Image decoding method and apparatus using same

ABSTRACT

An image decoding method includes: receiving information on a set of reference pictures for configuring a set of reference pictures of a current picture, wherein the information on the set of reference pictures includes MSB that may calculate the MSB of POC of a long-term reference picture relative to the current picture, and flag information that represents whether there is MSB; and eliciting the set of reference pictures by using received MSB when the flag information is 1, and performing marking on the reference picture, wherein the flag information may be 1 when a temporal sub-layer identifier is 0, and there is at least one POC value for which a remainder obtained by dividing by a maximum value MaxPicOrderCntLsb is the same as LSB of the POC of the long-term reference picture, in a set of POCs of a previous picture including POC values related to the previous picture.

This application is a National Stage Application of International PatentApplication No. PCT/KR2013/010632, filed on Nov. 21, 2013, which claimsthe benefit of Provisional Application Nos. 61/729,313, filed on Nov.21, 2012, 61/752,410 filed Jan. 14, 2013, 61/752,924 filed Jan. 15,2013, 61/753,868 filed Jan. 17, 2013, and 61/754,620 filed Jan. 20,2013, all of which are incorporated herein by reference in theirentirety.

TECHNICAL FIELD

The present invention relates to a video compression technique, and moreparticularly, to a method and an apparatus for signaling referencepicture list information.

BACKGROUND ART

Recently, demands for high-resolution and high-quality pictures haveincreased in various fields of applications. As pictures have higherresolution and higher quality, the amount of information on the picturesalso increases.

With a growing amount of information, multi-functional devices andnetworks with various environments are introduced. Accordingly, the samecontent may be utilized with different levels of quality.

Specifically, as terminals are able to support diverse qualities ofpictures and various network environments are established, a picturewith general quality is enabled in one environment while ahigher-quality picture may be available in another environment.

For example, a user may enjoy video content purchased through a portableterminal on a large-screen display with higher resolution at home.

In recent years, as high definition (HD) broadcast services areavailable, a large number of users are getting used to high-resolutionand high-quality videos and service providers and service users also payattention to ultrahigh-definition (UHD) services having a resolutionfour times higher than HDTV.

Thus, there is a need to provide scalability to video quality, forexample, the image quality, resolution, size and frame rate of a video,based on high-efficiency encoding and decoding methods on ahigh-capacity video so as to offer varied qualities of video services indifferent environments for users' demands.

DISCLOSURE Technical Problem

An aspect of the present invention is to provide a video decoding methodcapable of constructing a long-term reference picture set when anon-reference picture is discarded, and an apparatus using the same.

Another aspect of the present invention is to provide a video decodingmethod providing a condition for receiving LSB information forconstructing a long-term reference picture set, and an apparatus usingthe same.

Technical Solution

An embodiment of the present invention provides a video decoding methodthat decodes a bit stream, the method including receiving referencepicture set (RPS) information for constructing an RPS for a currentpicture, the RPS information comprising most significant bit (MSB)information for calculating an MSB of a picture order count (POC) of along-term reference picture (LTRP) for the current picture and flaginformation indicating whether the MSB information is present; andderiving the RPS using the received MSB information and marking areference picture when the flag information is 1, the flag informationequal to 1 if there is more than one POC in a previous picture POC setcomprising POCs relating to a previous picture that has a temporalsub-layer identifier equal to 0 and cannot be discarded withoutaffecting decodability of another picture in the same temporal layer forwhich the POC modulo MaxPicOrderCntLsb is equal to an LSB of the POC ofthe LTRP, MaxPicOrderCntLsb is the max least significant bit (LSB)value.

A network abstraction layer (NAL) unit type of the previous picture maybe none of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N, RSV_VCL_N10,RSV_VCL_N12, and RSV_VCL_N14.

The previous picture may not be a sub-layer non-reference pictureunavailable for inter prediction in decoding a next picture in decodingorder in the same sub-layer.

The previous picture POC set may include a POC of the previous picture,a POC of each reference picture in an RPS for the previous picture, anda POC of each picture following the previous picture and preceding thecurrent picture in decoding order.

The MSB information may be MSB cycle information determining a valuecorresponding to a difference in POC MSB between the current picture andthe LTRP.

The method may further include constructing a POC list for deriving theRPS, wherein the POC list comprises a short-term reference picture(STRP) POC list comprising a POC of an STRP for the current picture andan LTRP POC list comprising the POC of the LTRP for the current picture,the STRP POC list is generated using a difference in POC between thecurrent picture and the STRP, and the LTRP POC list is generated by anoperation that is POC of currentpicture−DeltaPocMsbCycleLt*MaxPicOrderCntLsb−POC LSB of currentpicture+POC LSB of LTRP, DeltaPocMsbCycleLt having a value correspondingto a difference in POC MSB between the current picture and the LTRP.

The deriving the RPS and marking the reference picture may includeconstructing an LTRP set using a picture with a POC LSB or POC the sameas a POC in the LTRP POC list an when the picture is present in adecoded picture buffer (DPB) storing a reconstructed picture; markingall pictures in the LTRP set as “used for long-term reference;”constructing an STRP set using a picture with a POC the same as a POC inthe STRP POC list when the picture is present in the DBP; and markingall reference pictures in the DPB which are not included on the LTRP setand the STRP set as “unused for reference.”

The marking the reference picture may mark the reference picture as“unused for reference” or “used for long-term reference.”

The method may include receiving slice type information on the currentpicture and prediction mode information on the current picture;determining based on the prediction mode information whether aprediction target block is subjected to intra prediction or interprediction; deriving a reference picture list based on the RPS wheninter prediction is applied to the prediction target block; deriving apredictive sample value of the prediction target block using thereference picture list; and generating a reconstructed picture based onthe predictive sample value of the prediction target block.

The method may further include marking the reconstructed picture as“used for short-term reference.”

Advantageous Effects

An embodiment of the present invention provides a video decoding methodcapable of constructing a long-term reference picture set when anon-reference picture is discarded, and an apparatus using the same.

Another embodiment of the present invention provides a video decodingmethod providing a condition for receiving LSB information forconstructing a long-term reference picture set, and an apparatus usingthe same.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram schematically illustrating a video encodingapparatus according to an exemplary embodiment of the present invention.

FIG. 2 is a block diagram schematically illustrating a video decodingapparatus according to an exemplary embodiment of the present invention.

FIG. 3 schematically illustrates a candidate block available when interprediction is performed on a current block according to an exemplaryembodiment.

FIG. 4 is a flowchart illustrating a process of marking a referencepicture marking.

FIG. 5 illustrates a process of deriving and marking a reference pictureset (RPS) according to one exemplary embodiment.

FIG. 6 illustrates a process of deriving and marking an RPS when anon-reference picture is discarded.

FIG. 7 illustrates a process of deriving an RPS and marking a picturefor Pic 266 and Pic 268 according to a first embodiment of the presentinvention.

FIG. 8 illustrates a marking process according to a third embodiment ofthe present invention.

FIG. 9 illustrates a marking process according to a fourth embodiment ofthe present invention.

FIG. 10 illustrates a marking process according to a fifth embodiment ofthe present invention.

FIG. 11 is a flowchart illustrating a process of deriving an RPS andmarking a picture according to the present invention.

FIG. 12 illustrates a method of deriving a picture order count (POC) ofa long-term reference picture (LTRP).

FIG. 13 is a flowchart illustrating a video decoding method according tothe present invention.

MODE FOR INVENTION

The present invention may be changed and modified variously and beillustrated with reference to different exemplary embodiments, some ofwhich will be described in detail and shown in the drawings. However,these embodiments are not intended for limiting the invention. Theterminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting the technical ideaof the invention. As used herein, the singular forms “a,” “an” and “the”are intended to include the plural forms as well, unless the contextclearly indicates otherwise. It will be further understood that theterms “include” and/or “have,” when used in this specification, specifythe presence of stated features, integers, steps, operations, elements,components, and/or combinations thereof, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or combinations thereof.

Although elements illustrated in the drawings are independently shownfor convenience of description of different distinctive functions in thevideo encoding apparatus/decoding apparatus, such a configuration doesnot indicate that each element is constructed by a separate hardwareconstituent or software constituent. That is, at least two elements maybe combined into a single element, or a single element may be dividedinto a plurality of elements to perform functions. It is to be notedthat embodiments in which some elements are integrated into one combinedelement and/or an element is divided into multiple separate elements areincluded in the scope of the present invention without departing fromthe essence of the present invention.

Hereinafter, exemplary embodiments of the invention will be described indetail with reference to the accompanying drawings. Like referencenumerals in the drawings refer to like elements throughout, andredundant descriptions of like elements will be omitted herein.

FIG. 1 is a block diagram schematically illustrating a video encodingapparatus according to an exemplary embodiment of the present invention.A scalable video encoding/decoding method or apparatus may be realizedby extension of a general video encoding/decoding method or apparatusthat does not provide scalability, and a scalable video encodingapparatus may be configured based on the video encoding apparatus FIG.2.

Referring to FIG. 1, the video encoding apparatus 100 includes a picturepartition module 105, a prediction module 110, a transform module 115, aquantization module 120, a rearrangement module 125, an entropy encodingmodule 130, a dequantization module 135, an inverse transform module140, a filter module 145 and a memory 150.

The picture partition module 105 may divide an input picture into atleast one block as a processing unit. Here, the block as the processingunit may be a prediction unit (PU), a transform unit (TU) or a codingunit (CU).

Processing unit blocks divided by the picture partition module 105 mayhave a quadtree structure.

The prediction module 110 may include an inter prediction module toperform inter prediction and an intra prediction module to perform intraprediction, which will be described. The prediction module 110 generatesa prediction block by performing prediction on the processing unit ofthe picture from the partition module 105. The processing unit of thepicture in the prediction module 110 may be a CU, a TU or a PU.Furthermore, the prediction module 110 may determine whether predictionperformed on the processing unit is inter prediction or intraprediction, and may determine details (for example, a prediction mode)of each prediction method. Here, a processing unit for performingprediction may be different from a processing unit for determining aprediction method and details on the prediction method. For example, aprediction method and a prediction mode may be determined by PU, whileprediction may be performed by TU.

In inter prediction, a prediction block may be generated by performingprediction based on information on at least one of previous and/orsubsequent pictures of a current picture. In intra prediction, aprediction block may be generated by performing prediction based oninformation on a pixel within the current picture.

A skip mode, a merge mode or a motion vector prediction (MVP) mode maybe used as an inter prediction method. In inter prediction, a referencepicture for a PU may be selected, and a reference block corresponding tothe PU may be selected. The reference block may be selected as a unit ofinter pixel. Subsequently, a prediction block having a minimum residualsignal from the current PU and a minimum-size motion vector isgenerated.

The prediction block may be generated as an integer sample unit or as afractional pixel unit, such as a ½ pixel unit and a ¼ pixel unit. Here,the motion vector may be also represented in a fractional pixel.

Information including an index of the reference pixel selected in interprediction, the motion vector (e.g., a motion vector predictor) and theresidual signal, is entropy-encoded and transferred to a decodingapparatus. In the skip mode, since the prediction block may be areconstructed block, the residual may not be generated, transformed,quantized and transferred.

In intra prediction, a prediction mode is determined by PU, andprediction may be performed by PU. Alternatively, a prediction mode maybe determined by PU, and intra prediction may be performed in TU.

Intra prediction may include 33 directional prediction modes and two ormore non-directional modes. The non-directional modes may include a DCprediction mode and a planar mode.

In intra prediction, the prediction block may be generated afterapplying a filter to a reference sample. Here, whether to apply thefilter to the reference sample may be determined on an intra predictionmode and/or size of a current block.

A residual value (residual block or residual signal) between thegenerated prediction block and an original block is input to thetransform module 115. Also, information on a prediction mode and amotion vector used for prediction are encoded along with the residualvalue by the entropy encoding module 130 and transferred to the decodingapparatus.

The transform module 115 transforms the residual block by a TU andgenerates a transform coefficient.

A transform block is a rectangular block of samples to which the sametransformation is applied. The transform block may be a TU and have aquadtree structure.

The transform module 115 may perform transformation based on aprediction mode applied to the residual block and a size of the block.

For example, when intra prediction is applied to the residual block andthe block has a 4×4 residual array, the transform module 115 maytransform the residual block using discrete cosine transform (DCT).Otherwise, the transform module 115 may transform the residual blockusing discrete sine transform (DST).

The transform module 115 may generate a transform block of transformcoefficients by transformation.

The quantization module 120 may quantize residual values transformed bythe transform module 115, that is, the transform coefficients, togenerate quantized transform coefficients. The coefficients generated bythe quantization module 120 are provided to the dequantization module135 and the rearrangement module 125.

The rearrangement module 125 rearranges the quantized transformcoefficients provided by the quantization module 120. Rearranging thequantized transform coefficients may enhance encoding efficiency in theentropy encoding module 130.

The rearrangement module 125 may rearrange a two-dimensional (2D) blockof the quantized transform coefficients into a one-dimensional (1D)vector using coefficient scanning.

The entropy encoding module 130 may entropy-encode symbols according toprobability distribution based on the quantized transform coefficientsrearranged by the rearrangement module 125 or encoding parameter valuesderived in coding, thereby outputting a bit stream. Entropy encoding isa method of receiving symbols having different values and representingthe symbols as a decodable binary sequence or string while removingstatistical redundancy.

Here, a symbol means a syntax element as an encoding/decoding target, acoding parameter, a value of a residual signal, or the like. A codingparameter, which is a parameter necessary for encoding and decoding, mayinclude information encoded by the encoding apparatus and transferred tothe decoding apparatus, such as a syntax element, and information to beinferred during an encoding or decoding process and means informationnecessary for encoding and decoding a picture. The coding parameter mayinclude, for example, values or statistics of an intra/inter predictionmode, a movement/motion vector, a reference picture index, an encodingblock pattern, presence or absence of a residual signal, a quantizedtransform parameter, a block size and block partition information. Aresidual signal may denote a difference between an original signal and aprediction signal, a signal obtained by transforming the differencebetween the original signal and the prediction signal, or a signalobtained by transforming and quantizing the difference between theoriginal signal and the prediction signal. The residual signal may bereferred to as a residual block in a block unit.

When entropy encoding is applied, symbols are represented such that asymbol having a high probability is allocated a small number of bits anda symbol having a low probability is allocated a large number of bits,thereby reducing a size of bit strings for symbols to be encoded.Accordingly, entropy encoding may enhance compression performance ofvideo encoding.

For entropy encoding, encoding methods, such as exponential Golomb,context-adaptive variable length coding (CAVLC) and context-adaptivebinary arithmetic coding (CABAC), may be used. For example, the entropyencoding module 130 may store a table used for performing entropyencoding, such as a variable length coding/code (VLC) table, and performentropy encoding using the stored VLC table. In addition, the entropyencoding module 130 may derive a binarization method for a target symboland a probability model for a target symbol/bin and perform entropyencoding using the derived binarization method or probability model.

Here, binarization means representing values of symbols as a binsequence/string. A bin means each bin value (0 or 1) when a symbol isrepresented as a bin sequence/string through binarization.

A probability model means a predicted probability of a symbol/bin as anencoding/decoding target that may be derived through contextinformation/context model. Context information/context model isinformation for determining a probability of a symbol/bin as anencoding/decoding target.

In more detail, CABAC as an entropy encoding method transforms a symbolthat is not binarized into a bin by binarization, determines a contextmodel using encoding information on a neighboring block and a block tobe encoded or information on a symbol/bin encoded in a previous stage,and predicts a probability of a bin according to the determined contextmodel to perform arithmetic encoding of the bin, thereby generating abit stream. Here, CABAC may determine the context model, and then updatethe context model using information on an encoded symbol/bin for acontext model for a next symbol/bin.

If necessary, the entropy encoding module 130 may make a modification toa received parameter set or syntax.

The dequantization module 135 dequantizes the values quantized by thequantization module 120, that is, the quantized transform coefficients,and the inverse transform module 140 inverse-transforms the valuesdequantized by the dequantization module 135.

The residual values generated through the dequantization module 135 andthe inverse transform module 140 are merged with the prediction blockpredicted by the prediction module 110, thereby generating areconstructed block.

FIG. 1 illustrates that a reconstructed block is generated by merging aresidual block with a prediction block through an adder. Here, the addermay be regarded as a separate module for generating a reconstructedblock (reconstructed block generation module).

The filter module 145 may apply a deblocking filter, an adaptive loopfilter (ALF), and a sample adaptive offset (SAO) to a reconstructedpicture.

The deblocking filter may remove block distortion generated onboundaries between blocks in the reconstructed picture. The ALF mayperform filtering based on a value obtained by comparing thereconstructed picture obtained by filtering blocks using the deblockingfilter with the original picture. The ALF may be employed only for highefficiency. The SAO reconstructs an offset difference between theresidual block to which the deblocking filter has been applied and theoriginal picture by a pixel unit, in which a band offset or an edgeoffset is used.

Meanwhile, the filter module 145 may not apply filtering to areconstructed block used in inter prediction.

The memory 150 may store the reconstructed block or picture obtained viathe filter module 145. The reconstructed block or picture stored in thememory 150 may be provided to the prediction module 110 performing interprediction.

FIG. 2 is a block diagram schematically showing a video decodingapparatus according to an exemplary embodiment of the present invention.As described above in FIG. 1, a scalable video encoding/decoding methodor apparatus may be realized by extension of a general videoencoding/decoding method or apparatus that does not provide scalability,and a scalable video decoding apparatus may be configured based on thevideo decoding apparatus FIG. 2.

Referring to FIG. 2, the video decoding apparatus 200 may include anentropy decoding module 210, a rearrangement module 215, andequantization module 220, an inverse transform module 225, a predictionmodule 230, a filter module 235, and a memory 240.

When a video bit stream is input from the video encoding apparatus, theinput bit stream may be decoded according to the same procedure forprocessing video information as used by the video encoding apparatus.

The entropy decoding module 210 entropy-decodes the input bit streamaccording to probability distribution to generate symbols including asymbol in a quantized coefficient form. Entropy decoding is a method ofreceiving a binary sequence or string and generating each symbol.Entropy decoding is similar to entropy encoding described above.

For example, if the video encoding apparatus uses variable length coding(VLC), such as CAVLC, to perform entropy encoding, the entropy decodingmodule 210 may perform entropy decoding by configuring the same VLCtable as used in the encoding apparatus. Furthermore, if the videoencoding apparatus uses CABAC to perform entropy ending, the entropydecoding module 210 may also perform entropy decoding using CABAC.

In more detail, CABAC as an entropy decoding method may receive a bincorresponding to each syntax element in the bit stream, determines acontext model using information on a syntax element to be decoded anddecoding information on a neighboring block and a block to be decoded orinformation on a symbol/bin decoded in a previous stage, and predict aprobability of a bin according to the determined context model toperform arithmetic decoding of the bin, thereby generating a symbolcorresponding to a value of each syntax element. Here, CABAC maydetermine the context model, and then update the context model usinginformation on a decoded symbol/bin for a context model for a nextsymbol/bin.

When entropy decoding is applied, symbols are represented such that asymbol having a high probability is allocated a small number of bits anda symbol having a low probability is allocated a large number of bits,thereby reducing a size of bit strings for symbols to be encoded.Accordingly, entropy decoding may enhance compression performance ofvideo decoding.

Information for generating a prediction block, among pieces ofinformation decoded by the entropy decoding module 210, may be providedto the prediction module 230. Residual values entropy-decoded by theentropy decoding module 210, that is, quantized transform coefficients,may be input to the rearrangement module 215.

The rearrangement module 215 may rearrange information on the bit streamentropy-decoded by the entropy decoding module 210, that is, thequantized transform coefficients, based on a rearrangement method usedin the encoding apparatus.

The rearrangement module 215 may reconstruct and rearrange a 1D vectorof coefficients into a 2D block of coefficients. The rearrangementmodule 215 may scan coefficients based on a prediction mode of a currentblock (transform block) and a size of the transform block to generate a2D block of coefficients (quantized transform coefficients).

The dequantization module 220 may perform dequantization based on aquantization parameter provided from the encoding apparatus and therearranged coefficients of the block.

The inverse transform module 225 may perform inverse DCT and/or inverseDST on a result of quantization performed by the video encodingapparatus in response to DCT and DST performed by the transform moduleof the encoding apparatus.

Inverse transformation may be performed on the basis of a transfer unitor a partition unit of a picture determined by the video encodingapparatus. The transform module of the video encoding apparatus mayselectively perform DCT and/or DST depending on a plurality ofinformation elements, such as a prediction method, a size of the currentblock and a prediction direction, and the inverse transform module 225of the video decoding apparatus may perform inverse transformation onthe basis of information on the transformation performed by thetransform module of the video encoding apparatus.

The prediction module 230 may generate a prediction block based oninformation on generation of the prediction block provided from theentropy decoding module 210 and information on a previously decodedblock and/or picture provided by the memory 240.

When a prediction mode for a current PU is an intra prediction mode,intra prediction may be performed based on information on a pixel in acurrent picture to generate the prediction block.

When a prediction mode for the current PU is an inter prediction mode,inter prediction for the current PU may be performed based oninformation on at least one of previous and subsequent pictures of thecurrent picture. Here, motion information necessary for the interprediction for the current PU provided by the video encoding apparatus,for example, information on a motion vector and a reference pictureindex, may be derived by checking a skip flag and a merge flag receivedfrom the encoding apparatus.

In inter prediction for the current picture, the prediction block with aminimum residual signal from the current block and a minimum-size motionvector may be generated.

Meanwhile, methods of deriving motion information may vary according toa prediction mode of the current block. An advanced motion vectorpredictor (AMVP) mode, a merge mode, or the like may be used as aprediction mode for inter prediction.

For example, when the AMVP mode is employed, the encoding apparatus andthe decoding apparatus may generate a motion vector candidate list byusing a motion vector of the reconstructed neighboring block and/or amotion vector of the collocated block. That is, the motion vector of thereconstructed neighboring block and/or the motion vector of thecollocated block may be used as motion vector candidates. The encodingapparatus may transmit a prediction motion vector index indicating anoptimal motion vector selected among the motion vector candidatesincluded in the list to the decoding apparatus. In this case, thedecoding apparatus may select a prediction motion vector of the currentblock, using the motion vector index, among the motion vector candidatesincluded in the motion vector candidate list.

The encoding apparatus may calculate a motion vector difference (MVD)between a motion vector of the current block and the prediction motionvector, encode the MVD and transmit the MVD to the decoding apparatus.Here, the decoding apparatus may decode the received MVD and add the MVDto the prediction motion vector to obtain the motion vector of thecurrent block.

The encoding apparatus may also transmit the reference picture indexindicating the reference picture to the decoding apparatus.

The decoding apparatus may predict the motion vector of the currentblock using motion information on neighboring blocks and derive themotion vector of the current block using a residual received from theencoding apparatus. The decoding apparatus may generate the predictionblock for the current block based on the derived motion vector andinformation of the reference picture index received from the encodingapparatus.

Alternatively, when the merge mode is employed, the encoding apparatusand the decoding apparatus may generate a merge candidate list usingmotion information on the reconstructed neighboring block and/or motioninformation on the collocated block. That is, when the motioninformation on the reconstructed neighboring block and/or on thecollocated block is present, the encoding apparatus and the decodingapparatus may use the motion information as a merge candidate for thecurrent block.

The encoding apparatus may select a merge candidate which providesoptimal coding efficiency among merge candidates included in the mergecandidate list as motion information for the current block. In thiscase, a merge index indicating the selected merge candidate may beincluded in a bit stream to be transmitted to the decoding apparatus.The decoding apparatus may select one of the merge candidates includedin the merge candidate list using the transmitted merge index anddetermine the selected merge candidate as the motion information for thecurrent block. Thus, when the merge mode is employed, the motioninformation on the reconstructed neighboring block and/or on thecollocated block may be used as the motion information for the currentblock as it is. The decoding apparatus may reconstruct the current blockby adding the prediction block to the residual transmitted from theencoding apparatus.

In the aforementioned AMVP and merge modes, the motion information onthe reconstructed neighboring block and/or motion information oncollocated block may be used in order to derive the motion informationon the current block.

In the skip mode as another mode used for inter prediction, informationon a neighboring block may be used for the current block as it is.Accordingly, in the skip mode, the encoding apparatus does not transmitsyntax information, such as a residual, to the decoding apparatus,except for information indicating which block motion information to beused as the motion information on the current block.

The encoding apparatus and the decoding apparatus may perform motioncompensation on the current block based on the derived motioninformation, thereby generating the prediction block of the currentblock. Here, the prediction block may refer to a motion-compensatedblock generated by performing motion compensation on the current block.Further, a plurality of motion-compensated blocks may form onemotion-compensated picture.

A reconstructed block may be generated using the prediction blockgenerated by the prediction module 230 and the residual block providedby the inverse transform module 225. FIG. 2 illustrates that thereconstructed block is generated by the adder merging the predictionblock with the residual block. Here, the adder may be regarded as aseparate module for generating the reconstructed block (reconstructedblock generation module).

When the skip mode is used, the prediction block may be thereconstructed block without transmitting the residual block.

The reconstructed block and/or picture may be provided to the filtermodule 235. The filter module 235 may apply deblocking filtering, SAOand/or AFL to the reconstructed block and/or picture.

The memory 240 may store the reconstructed picture or block to be usedas a reference picture or a reference block and supply the reconstructedpicture to an output unit.

Components directly related to video decoding among the entropy decodingmodule 210, the rearrangement module 215, the dequantization module 220,the inverse transform module 225, the prediction module 230, the filtermodule 235 and the memory 240 of the decoding apparatus 200, forexample, the entropy decoding module 210, the rearrangement module 215,the dequantization module 220, the inverse transform module 225, theprediction module 230 and the filter module 235 may be defined as adecoder or a decoding unit, separately from the other components.

Further, the decoding apparatus 200 may further include a parsing module(not shown) to parse information about an encoded video included in thebit stream. The parsing module may include the entropy decoding module210 or be included in the entropy decoding module 210. The parsingmodule may be provided as one component of the decoding unit.

FIG. 3 schematically illustrates a candidate block available when interprediction is performed on a current block according to an exemplaryembodiment.

The prediction modules of the encoding apparatus and the decodingapparatus may use a block at a preset position neighboring a currentblock 300 as a candidate block. Referring to FIG. 3 as an example, twoblocks A₀ 310 and A₁ 320 at bottom left positions of the current blockand three blocks B₀ 330, B₁ 340 and B₂ 350 at top right and top leftpositions of the current block may be selected as spatial candidateblocks. In addition to the spatially neighboring blocks, a COL block 360may be used as a temporal candidate block. The COL block 360 may be ablock in a reconstructed collocated (col) picture spatiallycorresponding to the current block or a block present at a predeterminedrelative position (position inside and/or outside the block spatiallycorresponding to the current block) in the reconstructed collocated(col) picture.

In inter prediction, prediction of the current block may be performedbased on a reference picture, which is at least one of previous andsubsequent pictures of a current picture. A picture used for generatinga prediction block of the current block, that is, for prediction of thecurrent block is referred to as a reference picture or reference frame.

A reference picture for the current block may be derived from areference picture for a neighboring block or indicated by informationreceived from the encoding apparatus. In the skip mode or merge mode,the prediction module of the decoding apparatus may use the referencepicture for the neighboring block as the reference picture for thecurrent picture. When the MVP mode is applied, the prediction module ofthe decoding apparatus may receive information indicating the referencepicture for the current block from the encoding apparatus.

The reference picture is specified by a reference picture index refIdx,and a predetermined region in the reference picture is specified by amotion vector.

Pictures encoded/decoded prior to a current picture may be stored in amemory, for example, a decoded picture buffer (DPB), and be used forprediction of the current block or current picture. Pictures availablefor inter prediction of the current block may be maintained in areference picture list. Here, a reference picture used for interprediction of the current block among the reference pictures included inthe reference picture list may be indicated by a reference pictureindex. That is, the reference picture index may refer to an indexindicating the reference picture used for inter prediction of thecurrent block among the reference pictures forming the reference picturelist.

A P slice is a slice decoded by intra prediction or by inter predictionusing at most one motion vector and one reference picture. A B slice isa slice decoded by intra prediction or by inter prediction using at mosttwo motion vectors and two reference pictures. Here, the referencepictures may include short-term reference pictures (STRPs) and long-termreference pictures (LTRPs). Pictures may be specified by Picture OrderCount (POC) which represents display order, in which STRPs may bepictures having an small difference in POC from the current picture andLTRPs may be pictures having a large difference in POC from the currentpicture. Hereinafter, POC may be used to include a POC value.

Reference picture list 0 (“L0”) is a reference picture list used forinter prediction of a P slice or B slice. Reference picture list 1(“L1”) is used for inter prediction of a B slice. Thus, L0 is used forinter prediction of a block of a P slice involved in unidirectionalprediction, while L0 and L1 are used for inter prediction of a block ofa B slice involved in bidirectional prediction.

The reference picture list may be constructed based on a referencepicture set (RPS) determined or generated by the encoding apparatus andthe decoding apparatus. An RPS refers to an available reference picturepresent in a decoding memory, for example in the DPB, for decoding aslice or picture. Information relating to an RPS needed for decoding aslice (and/or picture) may be transmitted through a sequence parameterset (SPS), a picture parameter set (PPS) and/or a slice header.

The RPS may include reference pictures used for reference for thecurrent picture/slice or a future picture/slice. Reference pictures usedfor decoding a slice and/or picture may include STRPs and LTRPs.Further, STRPs may include forward STRPs having a lower POC than that ofthe current picture and backward STRPs having a higher POC than that ofthe current picture. Here, an RPS may be determined or generated withrespect to each of forward STRPs, backward STRPs and LTRPs.

To identify STRPs, differences in POC between the current picture andthe STRPs are signaled. For an LTRP, a least significant bit (LSB) of aPOC of the LTRP is signaled.

When an LTRP is signaled through a slice header, if at least tworeference pictures having the same POC LSB are present in the DPB, oneof which is an LTRP, additional information for determining a mostsignificant bit (MSB) of a POC of the LTRP may need to be signaled forclear signaling.

Meanwhile, before decoding a slice, five lists including POC values mayneed constructing to derive an RPS. The five lists includePocStCurrBefore, PocStCurrAfter, PocStFoll, PocLtCurr and PocLtFoll. Thelists may include numbers of components, that is, POC values, thenumbers being specified by NumPocStCurrBefore, NumPocStCurrAfter,NumPocStFoll, NumPocLtCurr and NumPocLtFoll, respectively.

The lists are described in brief as below.

(1) PocStCurrBefore: A list including a POC of an STRP which is used bythe current picture and has a smaller POC than that of the currentpicture

(2) PocStCurrAfter: A list including a POC of an STRP that is used bythe current picture and has a larger POC than that of the currentpicture

(3) PocStFoll: A list including a POC of an STRP not used by the currentpicture

(4) PocLtCurr: A list including a POC of an LTRP that is used by thecurrent picture

Here, when no MSB information is signaled, for example,delta_poc_msb_present_flag [i] is 0, a POC listed in PocLtCurrcorresponds only to an LSB of a POC of a reference picture.

delta_poc_msb_present_flag is flag information indicating whetherdelta_poc_msb_cycle_lt[i] is present, and delta_poc_msb_cycle_lt[i] is avalue for determining DeltaPocMsbCycleLt used for calculating a POC MSBof an LTRP in a long-term reference picture set (LTRPS) of the currentpicture. DeltaPocMsbCycleLt may correspond to a difference between anMSB of the POC of the current picture and an MSB of a POC of a referencepicture.

(5) PocLtFoll: A list including a POC of an LTRP not used by the currentpicture

Here, when no MSB information is signaled, for example,delta_poc_msb_present_flag [i] is 0, a POC listed in PocLtFollcorresponds only to an LSB of a POC of a reference picture.

Based on the foregoing information, a process of deriving an RPS andmarking a picture may be carried out as follows. Each reference pictureis derived as components for an RPS and marked via an iterative specificprocess corresponding to a number of reference pictures.

Hereinafter, for convenience of description, a reference picture set offorward STRPs is defined as a forward STRP set (“STRPS”), a referencepicture set of backward STRPs as a backward STRP set (“STRPS”), and areference picture set of LTRPs as an LTRP set (“LTRPS”). For example,the forward STRPS may be represented as RefPicSetStCurrBefore, thebackward STRPS as RefPicSetStCurrAfter, and the LTRPS asRefPicSetLtCurr.

Further, an STRPS not used by the current picture may be represented asRefPicSetStFoll, and an LTRPS not used by the current picture asRefPicSetLtFoll.

1. An LTRPS is derived first, and an LTRP is marked.

for( i = 0; i < NumPocLtCurr; i++ )↓ if( !CurrDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB ↓ withpic_order_cnt_lsb equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[ i ] =picX↓ else if( there is a short-term reference picture picY in the DPBwith pic_order_cnt_lsb equal to PocLtCur[ i ] )↓ RefPicSetLtCurr[ i ] =picY↓ else ↓ RefPicSetLtCurr[ i ] = “no reference picture”↓ else↓ if(there is a long-term reference picture picX in the DPB↓ withPicOrderCntVal equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[ i ] = picX↓else if( there is a short-term reference picture picY in the DPB withPicOrderCntVal equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[ i ] = picY↓else ↓ RefPicSetLtCurr[ i ] = “no reference picture” 

for( i = 0; i < NumPocLtFoll; i++ )↓ if( !FollDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓else if( there is a short-term reference picture picY in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picY↓else ↓ RefPicSetLtFoll[i ] = “no reference picture”↓ else↓ if( there isa long-term reference picture picX in the DPB↓ with PicOrderCntVal toPocLtFoll[i ] )↓ RefPicSetLtFoll[i ] = picX↓ else if( there is ashort-term reference picture picY in the DPB with PicOrderCntVal equalto PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ]=picY↓ else↓ RefPicSetLtFoll[i ]= “no reference picture” 

2. All reference pictures included in RefPicSetLtCurr andRefPicSetLtFoll are marked as “used for long-term reference,” that is,“used as LTRPs.”

3. Subsequently, an STRPS is derived, and an STRP is marked.

for( i = 0; i < NumPocStCurrBefore; i++ )↓ if( there is a short-termreference picture picX in the DPB↓ with PicOrderCntVal equal toPocStCurrBefore[i ])↓ RefPicSetStCurrBefore[i ] = picX↓ else↓RefPicSetStCurrBefore[i ] = “no reference picture” 

for( i = 0; i < NumPocStCurrAfter; i++ )↓ if( there is a short-termreference picture picX in the DPB↓ with PicOrderCntVal equal toPocStCurrAfter[i ])↓ RefPicSetStCurrAfter[ i ] = picX↓ else↓RefPicSetStCurrAfter[ i ] = “no reference picture” for( i = 0; i <NumPocStFoll; i++ )↓ if( there is a short-term reference picture picX inthe DPB↓ with PicOrderCntVal equal to PocStFoll[i ])↓ RefPicSetStFoll[i] = picX↓ else↓ RefPicSetStFoll[i ] = “no reference picture” 

4. All reference pictures present in the DPB, not included inRefPicSetLtCurr, RefPicSetLtFoll, RefPicSetStCurrBefore,RefPicSetStCurrAfter or RefPicSetStFoll are marked as “unused forreference,” that is, “unused as reference pictures.”

FIG. 4 is a flowchart illustrating a reference picture marking process.Each reference picture in the DPB is subjected to the following process.

First, it is checked whether a picture to be marked as a referencepicture, that is, a first reference picture, is included in an LTRPSRefPicSetLtCurr or in a non-reference LTRPS RefPicSetLtFoll. Thenon-reference LTRPS is not referred to by a current (S410).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, the first reference picture is marked as “used forlong-term reference” (S420), followed by operation S450.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, operation S430 is carried out.

It is checked whether the first reference picture is included in aforward STRP set RefPicSetStCurrBefore, in a backward STRP setRefPicSetStCurrAfter, or in a non-reference STRP set RefPicSetLtFoll(S430).

If the first reference picture is not included in the forward STRP set,in the backward STRP set, and in the non-reference STRP, the firstreference picture is marked as “unused for reference” (S440), followedby operation S450.

If the first reference picture is included in the forward STRP set, inthe backward STRP set, or in the non-reference STRP, operation S450 iscarried out.

In operation S450, it is determined whether more reference pictures arepresent in the DPB, and if additional reference pictures are present inthe DPB, a next reference picture as a reference picture marking targetis subjected to the foregoing process to start operation S410.

If a reference picture to be marked as a reference picture is notpresent, the reference picture marking process terminates.

The foregoing process of deriving and marking a reference picture setinvolves a robustness problem. That is, when deriving an RPS, such asRefPicSetLtCurr, RefPicSetLtFoll, RefPicSetStCurrBefore,RefPicSetStCurrAfter and RefPicSetStFoll, and marking a state of areference picture in the DPB, the foregoing process may be inappropriateunder a particular situation.

FIG. 5 illustrates a process of deriving and marking an RPS according toone exemplary embodiment. Referring to FIG. 5, pictures may beclassified into reference pictures and non-reference pictures. Thereference pictures are packetized into a TRAIL_R NAL unit, and thenon-reference pictures are packetized into a TRAIL_N NAL unit.

A temporal sub-layer access (TSA) picture as a TSA NAL unit type is apicture which is switchable between temporal sub-layers in a bit streamsupporting temporal scalability, indicating an up-switchable positionfrom a lower sub-layer to an upper sub-layer.

A step-wise temporal sub-layer access (STSA) picture as an STSA NAL unittype is a picture which is switchable between temporal sub-layers in abit stream supporting temporal scalability, indicating an up-switchableposition from a lower sub-layer to a one-level higher upper sub-layer.

A TRAIL picture as a TRAIL NAL unit type is a picture following arandomly accessible picture in an output order and decoding order.

As shown in FIG. 5, an NAL unit type, which represents a property of apicture, may be marked with whether the picture is referenced by adifferent picture. A picture with an NAL unit type marked as “_N” refersto a non-reference picture that a different picture does not refer to,and a picture with an NAL unit type marked as “_R” refers to a referencepicture that a different picture refers to.

When an NAL unit type is TRAIL_N, TSA_N or STSA_N, a decoded picture maynot be included in any of RefPicSetStCurrBefore, RefPicSetStCurrAfterand RefPicSetLtCurr of pictures having the same temporal sub-layer ID(TemporalId).

That is, a coding picture with an NAL unit type of TRAIL_N, TSA_N orSTSA_N may be discarded, without affecting decoding of pictures with thesame temporal sub-layer ID (TemporalId).

Referring to FIG. 5, a picture having a POC of 264 (“Pic 264”), apicture having a POC of 266 (“Pic 266”) and a picture having a POC of268 (“Pic 268”) have an NAL unit type of TRAIL_R and thus may bereferenced by a different picture. A picture having a POC of 265 (“Pic265”) and a picture having a POC of 267 (“Pic 267”) have an NAL unittype of TRAIL_N and thus may be non-referenced by another picture.

Suppose that when a first slice of Pic 266 is received, an LTRP having aPOC of 0 (“Pic 0) and three STRPs having POCs of 252, 256 and 264 (“Pic252,” “Pic 256” and “Pic 264”) are present in the DPB. When Pic 266, Pic267 and Pic 268 are received, a process of deriving an RPS and marking apicture is as follows.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows. Here, since the DPB includes pictures having the same POC LSB,for example, Pic 0 and Pic 256, a slice header necessarily signals MSBinformation on the LTRP. As an LSB of a POC is signaled using 8 bits,Pic 0 and Pic 256 have the same POC LSB.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 252 is marked as “unused for            reference”

2. For Pic 267, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={ }    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 266 is marked as “used for            Short-term reference”        -   Reference picture with POC 0 is marked as “unused for            reference”

3. For Pic 268, an RPS is derived and a reference picture is marked asfollows. Here, since the DPB includes only a single picture with a POCLSB of 0, the slice header may not need to signal the MSB information onthe LTRP.

-   -   RefPicSetLtCurr={256}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            long-term reference.”            ,        -   Reference picture with POC 264 is marked as “used for            Short-term reference”        -   Reference picture with POC 266 is marked as “used for            short-term reference”

FIG. 6 illustrates a process of deriving and marking an RPS when anon-reference picture is discarded.

FIG. 6 shows that the non-reference pictures Pic 265 and Pic 267 havingthe NAL unit type of TRAIL_N are discarded under the same condition asin FIG. 5. FIG. 6 involves mismatches in the process of deriving andmarking the RPS.

When the non-reference pictures are discarded, a process of deriving anRPS and marking a picture is as follows when Pic 266 and Pic 268 arereceived.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows, in which case no problem occurs.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked “as used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-tem reference”        -   Reference picture with POC 252 is marked as “unused for            reference”

2. For Pic 268, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked is “used for            long-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 266 is marked as “used for            short-term reference”        -   Reference picture with POC 256 is marked as “unused for            reference”

When deriving the RPS for Pic 268, a problem occurs. Although thereference picture with the POC of 256 is initially intended to be markedas “used for long-term reference,” the reference picture with the POC of256 is marked as “unused for reference” as illustrated above.

As in FIG. 6, when Pic 267 is discarded, information indicating that thereference picture with the POC of 0 is marked as “unused for reference”and the reference picture is necessarily discarded is not transmitted.Thus, the reference picture with the POC of 0 remains in the DPB until aprocess of marking a next picture. Since the reference picture with thePOC of 0 remains in the DPB, pictures having the same POC LSB, that is,the reference pictures with the POCs of 0 and the POCs of 256, arepresent in the DPB. However, the reference pictures having the same POCMSB are necessarily be not present in initially intended situation andMSB information is not signaled, causing a problem.

When an algorithm for deriving RefPicSetLtCurr and RefPicSetLtFoll isapplied, a problem occurs that a reference picture marked as “used forlong-term reference” is maintained and accordingly the reference picturewith the POC of 256 cannot but be marked as “unused for reference.”

In order to solve the foregoing problem, various exemplary embodimentsmay be applicable, which will be described below.

First Embodiment

According to a first embodiment of the present invention, informationfor calculating or inferring an MSB of a POC of an LTRP is signaledthrough a slice header.

In the present embodiment, the following method is used to signal anLTRP via a slice header. In this case, delta_poc_msb_present_flag [i]does not need signaling, and delta_poc_msb_cycle_lt [i] is alwayssignaled.

1. A process of deriving POC lists (PocLtCurr and PocLtFoll) of an LTRPis as follows.

for( i = 0, j = 0, k = 0; i < num_long_term_sps + num_long_term_pics;i++ ) {↓ pocLt = PocLsbLt[ i ] + (PicOrderCntVal − DeltaPocMSBCycleLt[ i]* MaxPicOrderCntLsb − pic_order_cnt_lsb) 

if( UsedByCurrPicLt[i ] ) {↓ PocLtCurr[j ] = pocLt↓ } else {↓ PocLtFoll[k ] = pocLt↓ }↓ }↓ NumPocLtCurr = j↓ NumPocLtFoll = k 

As described above, defining a number (NumPocLtCurr) of POCs included inPocLtCurr as j and a number (NumPocLtFoll) of POCs included in PocLtFollas k, the foregoing process is repeated j+k times, thereby calculating aPOC (pocLt) of the LTRP.

PocLsbLt[i] is a variable representing a POC LSB of an i-th LTRP,PicOrderCntVal is a variable representing a POC of a current picture,and DeltaPocMSBCycleLt[i] is a variable representing a difference in POCMSB between the current picture and the i-th LTRP.

MaxPicOrderCntLsb is a maximum value representing a POC LSB, andpic_order_cnt_lsb is a value representing a remainder obtained bydividing the POC of the current picture by MaxPicOrderCntLsb, which istransmitted from the encoding apparatus to the decoding apparatus.

2. An RPS for the LTRP is derived using PocLtCurr and PocLtFoll obtainedby the process 1 as follows.

for( i = 0; i < NumPocLtCurr; i++ )↓ if( there is a long-term referencepicture picX in the DPB↓ with PicOrderCntVal equal to PocLtCurr[ i ] )↓RefPicSetLtCurr[i ] = picX↓ else if( there is a short-term referencepicture picY in the DPB↓ with PicOrderCntVal equal to PocLtCurr[ i ] )↓RefPicSetLtCurr[i ] = picY↓ else ↓ RefPicSetLtCurr[i ] = “no referencepicture”↓ ↓ for( i = 0; i < NumPocLtFoll; i++ ) ↓ if( there is along-term reference picture picX in the DPB↓  with PicOrderCntVal toPocLtFoll[i ] )↓ RefPicSetLtFoll[i ] = picX↓ else if( there is ashort-term reference picture picY in the DPB↓  with PicOrderCntVal equalto PocLtFoll[i ] )↓ RefPicSetLtFoll[i ] = picY↓ else↓ RefPicSetLtFoll[i] = “no reference picture” 

A syntax element relating to the LTRP signalled via the slice header inthe present embodiment is illustrated in Table 1.

TABLE 1 Descriptor slice_segment_header( ) { ... if(!dependent_slice_segment_flag ) { ... if( !IdrPicFlag ) { ... if(long_term_ref_pics_present_flag ) { if( num_long_term_ref_pics_sps > 0 )num_long_term_sps ue(v) num_long_term_pics ue(v) for( i = 0; i <num_long_term_sps + num_long_term_pics; i++ ) { if( i <num_long_term_sps ) lt_idx_sps[ i ] u(v) else { poc_lsb_lt[ i ] u(v)used_by_curr_pic_lt_flag[ i ] u(1) } delta_poc_msb_cycle_lt[ i ] ue(v) }} } ... } ... }

Referring to Table 1, num_long_term_sps specifies a number of entries ofan LTRPS of the current picture derived based on a candidate LTRPspecified in an activated SPS. num_long_term_sps has a value rangingfrom 0 to a value specified by num_long_term_ref_pics_sps. Ifnum_long_term_sps is not present, num_long_term_sps is inferred as 0.

num_long_term_pics specifies a number of entries of an LTRPS of thecurrent picture directly signaled via the slice header. Ifnum_long_term_pic is not present, num_long_term_pic is inferred as 0.

lt_idx_sps[i] specifies an index of an i-th entry of the LTRPS of thecurrent picture in a list of the candidate LTRP specified by theactivated SPS. A bit number representing lt_idx_sps[i] is Ceil(Log2(num_long_term_ref_pics_sps). It lt_idx_sps[i] is not present,lt_idx_sps[i] is inferred as 0. lt_idx_sps[i] has a value ranging from 0to num_long_term_ref_pics_sps−1.

poc_lsb_lt[i] specifies the value of a POC modulo MaxPicOrderCntLsb ofthe i-th entry an LTRPS of the current picture. poc_lsb_lt[i] has alength of log 2_max_pic_order_cnt_lsb_minus4+4 bits.

used_by_curr_pic_lt_flag[i] equal to 0 specifies that the i-th entry ofthe LTRPS of the current picture is not referenced by the currentpicture.

delta_poc_msb_cycle_lt[i] is used to determine an MSB of a POC of thei-th entry of the LTRPS of the current picture. Ifdelta_poc_msb_cycle_lt is not present, delta_poc_msb_cycle_lt isinferred as 0.

FIG. 7 illustrates a process of deriving an RPS and marking a picturefor Pic 266 and Pic 268 illustrated in FIG. 6 according to the firstembodiment. According to the embodiment of FIG. 7, the foregoing problemdoes not occur any more.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 252 is marked as “unused for            reference”

2. For Pic 268, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={256}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            long-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 266 is marked as “used for            short-term reference”        -   Reference picture with POC 0 is marked as “unused for            reference”

As illustrated in Table 1, as MSB information on an LTRP for eachpicture is signaled, a picture with a POC of 256 is marked as “used forlong-term reference” and a picture with a POC of 0 is marked as “unusedfor reference” when the LTRPS for Pic 268 is derived.

Second Embodiment

According to a second embodiment of the present invention, wheninformation for calculating or inferring an MSB of a POC of an LTRP withrespect to a specific POC LSB is signaled once, the information forcalculating or inferring the MSB of the POC is always used along withthe specific POC LSB when the specific POC LSB is used for signaling theLTRP.

The second embodiment may be modified as follows.

Variation 1

When information for calculating or inferring an MSB of a POC of an LTRPwith respect to a specific POC LSB is signaled once, the information forcalculating or inferring the MSB of the POC is always used along withthe specific POC LSB when the specific POC LSB is used for signaling theLTRP with respect to a picture remaining in a sequence.

Variation 2

When information for calculating or inferring an MSB of a POC of an LTRPwith respect to a specific POC LSB is signaled once, the information forcalculating or inferring the MSB of the POC is always used along withthe specific POC LSB when the specific POC LSB is used for signaling theLTRP with respect to a picture remaining in a bit stream.

Variation 3

When information for calculating or inferring an MSB of a POC of an LTRPwith respect to a specific POC LSB is signaled once, the information forcalculating or inferring the MSB of the POC is always used along withthe specific POC LSB when the specific POC LSB is used for signaling theLTRP to a next random access point.

Derived RPSs and marked pictures for Pic 266 and Pic 268 of FIG. 6according to the present embodiment are illustrated in FIG. 7.

In FIG. 7, since MSB information is signaled for a picture with an LSBof a previous POC of 0, MSB information for an LTRP is necessarilysignaled with respect to the picture with the POC of 268. The MSBinformation is signaled even when there are no reference pictures havingthe same POC LSB in the DPB by discarding a picture, and accordingly theforegoing problem does not occur any more.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 252 is marked as “unused for            reference”

2. For Pic 268, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={256}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            long-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 266 is marked as “used for            short-term reference”        -   Reference picture with POC 0 is marked as “unused for            reference”

The Third Embodiment

According to a third embodiment of the present invention, if a currentpicture can be discarded without affecting decodability of otherpictures, an LTRP in the DPB is not allowed to be marked as “unused forreference.”

A process of deriving an RPS according to the present embodiment is asfollows.

1. An LTRPS is derived first, and an LTRP is marked.

for( i = 0; i < NumPocLtCurr; i++ )↓ if( !CurrDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB ↓ withpic_order_cnt_lsb equal to PocLtCurr[i ] )↓ RefPicSetLtCurr[i ] = picX↓else if( there is a short-term reference picture picY in the DPB withpic_order_cnt_lsb equal to PocLtCurr[i ] )↓ RefPicSetLtCurr[ i ] = picY↓else ↓ RefPicSetLtCurr[i ] = “no reference picture”↓ else↓ if( there isa long-term reference picture picX in the DPB↓ with PicOrderCntVal equalto PocLtCurr[ i ] )↓ RefPicSetLtCurr[i ] = picX↓ else if( there is ashort-term reference picture picY in the DPB↓ with PicOrderCntVal equalto PocLtCurr[ i ] )↓ RefPicSetLtCurr[i ] = picY↓ else ↓RefPicSetLtCurr[i ] = “no reference picture” 

for( i = 0; i < NumPocLtFoll; i++ )↓ if( !FollDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓else if( there is a short-term reference picture picY in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picY↓else ↓ RefPicSetLtFoll[i ] = “no reference picture”↓ else↓ if( there isa long-term reference picture picX in the DPB↓ with PicOrderCntVal toPocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓ else if( there is ashort-term reference picture picY in the DPB with PicOrderCntVal equalto PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picY↓ else↓ RefPicSetLtFoll[i] = “no reference picture” 

2. All reference pictures included in RefPicSetLtCurr andRefPicSetLtFoll are marked as “used for long-term reference,” that is,“used as LTRPs.”

3. As illustrated below, an STRPS is derived, and an STRP is marked.

for( i = 0; i < NumPocStCurrBefore; i++ )↓ if( there is a short-termreference picture picX in the DPB↓ with PicOrderCntVal equal toPocStCurrBefore[i ])↓ RefPicSetStCurrBefore[i ] = picX↓ else↓RefPicSetStCurrBefore[i ] = “no reference picture” 

for( i = 0; i < NumPocStCurrAfter; i++ )↓ if( there is a short-termreference picture picX in the DPB↓ with PicOrderCntVal equal toPocStCurrAfter[ i ])↓ RefPicSetStCurrAfter[ i ] = picX↓ else↓RefPicSetStCurrAfter[ i ] = “no reference picture” for( i = 0; i <NumPocStFoll; i++ )↓ if( there is a short-term reference picture picX inthe DPB↓ with PicOrderCntVal equal to PocStFoll[ i ])↓ RefPicSetStFoll[i ] = picX↓ else↓ RefPicSetStFoll[ i ] = “no reference picture” 

4. If an NAL unit type of the current picture is not a non-referencepicture, that is, TRAIL_N, TSA_N, STSA_N, RADL_N or RASL_N, allreference pictures in the DPB marked as “used for long-term reference,”not included in RefPicSetLtCurr and RefPicSetLtFoll, are marked as“unused for reference.”

A random access decodable leading (RADL) picture as a RADL NAL unit typerefers to a picture which precedes a random access point picture inoutput order but follows the random access point picture in decodingorder, which is not used as reference picture for decoding process oftrailing pictures associated the random access point picture.

A random access skipped leading (RASL) picture as an RASL NAL unit typerefers to a picture which precedes a random access point picture inoutput order but follows the random access point picture in decodingorder, which relates to a BLA picture and a CRA picture. The RASLpicture may refer to pictures that are not present in a bit stream andthus may not be output and decoded correctly.

5. Alternatively, if the NAL unit type of the current picture is any oneof TRAIL_N, TSA_N, STSA_N, RADL_N and RASL_N as non-reference pictures,all reference pictures in the DPB marked as “used for long-termreference,” not included in RefPicSetLtCurr and RefPicSetLtFoll, areautomatically included in RefPicSetLtFoll.

6. All reference pictures in the DPB marked as “used for long-termreference” are marked as “unused for reference,” not included inRefPicSetLtCurr, RefPicSetLtFoll, RefPicSetStCurrBefore,RefPicSetStCurrAfter and RefPicSetStFoll, are marked as “unused forreference.”

FIG. 8 illustrates a marking process according to the third embodiment.Each reference picture in the DBP may be marked according to the processof FIG. 8.

First, it is determined whether a first reference picture is marked asan STRP, that is, whether the first reference picture is an STRP (S810).

If the first reference picture is an STRP, operation S820 is carriedout. If the first reference picture is not an STRP, operation S870 iscarried out.

If the first reference picture is an STRP, it is determined whether thefirst reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll (S820).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, the first reference picture is marked as “used forlong-term reference” (S830), followed by operation S860.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, operation S840 is carried out.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, it is determined whether the first reference picture isincluded in RefPicSetStCurrBefore, RefPicSetStCurrAfter, orRefPicSetLtFoll (S840).

If the first reference picture is not included in RefPicSetStCurrBefore,RefPicSetStCurrAfter, and RefPicSetLtFoll, the first reference pictureis marked as “unused for reference” (S850), followed by operation S860.

If the first reference picture is included in RefPicSetStCurrBefore,RefPicSetStCurrAfter, or RefPicSetLtFoll, operation S860 is carried out.

If the first reference picture is not an STRP in operation S810, it isdetermined whether the first reference picture is included inRefPicSetLtCurr or RefPicSetLtFoll (S870).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, operation S860 is carried out. If the first referencepicture is not included in RefPicSetLtCurr and RefPicSetLtFoll,operation S880 is carried out.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, it is determined whether an NAL unit type of a currentpicture is a non-reference picture (S880).

If the NAL unit type of the current picture is a non-reference picture,the first reference picture is marked as “unused for reference” (S850),followed by operation S860.

If the NAL unit type of the current picture is not a non-referencepicture, operation S860 is carried out.

It is determined whether more reference pictures are present in the DPBin operation S860. If an additional reference picture is present, a nextreference picture as a reference picture marking target is subjected tothe foregoing process to start operation S810.

If there is no reference picture as a reference picture marking target,the reference picture marking process terminates.

Derived RPSs and marked pictures for Pic 266 and Pic 268 of FIG. 6according to the present embodiment are illustrated in FIG. 7.

When processing Pic 267 having an NAL unit type of TRAIL_N, an LTRP maynot be marked as “unused for reference,” and thus two reference pictureshaving the same POC LSB are present when processing Pic 268. Thus,according to FIG. 7, in a process of marking Pic 268, MSB informationfor an LTRP is signaled, and accordingly the foregoing problem does notoccur any more.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference Picture with POC 252 is marked as “unused for            reference”

2. For Pic 268, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={256}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            long-tem reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 266 is marked as “used or            short-tem reference”        -   Reference picture with POC 0 is marked as “unused or            reference”

Fourth Embodiment

According to a fourth embodiment of the present invention, if a currentpicture is not a base temporal layer, that is, TemporalId identifying atemporal sub-layer is not 0, an LTRP in the DPB is not allowed to bemarked as “unused for reference.”

The fourth embodiment may be varied as follows.

When the current picture is a base temporal layer, that is, TemporalIdis 0, an LTRP in the DPB is marked as “unused for reference.”

According to the foregoing variation of the fourth embodiment, an RPSfor an LTRP is derived as follows.

1. An LTRPS is derived first, and an LTRP is marked.

for( i = 0; i < NumPocLtCurr; i++ )↓ if( !CurrDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB ↓ withpic_order_cnt_lsb equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[ i ] =picX↓ else if( there is a short-term reference picture picY in the DPBwith pic_order_cnt_lsb equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[i ] =picY↓ else ↓ RefPicSetLtCurr[i ] = “no reference picture”↓ else↓ if(there is a long-term reference picture picX in the DPB↓ withPicOrderCntVal equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[ i ] = picX↓else if( there is a short-term reference picture picY in the DPB withPicOrderCntVal equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[i ] = picY↓else ↓ RefPicSetLtCurr[ i ] = “no reference picture” 

for( i = 0; i < NumPocLtFoll; i++ )↓ if( !FollDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓else if( there is a short-term reference picture picY in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picY↓else ↓ RefPicSetLtFoll[i ] = “no reference picture”↓ else↓ if( there isa long-term reference picture picX in the DPB↓ with PicOrderCntVal toPocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓ else if( there is ashort-term reference picture picY in the DPB with PicOrderCntVal equalto PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picY↓ else↓ RefPicSetLtFoll[i] = “no reference picture” 

2. All reference pictures included in RefPicSetLtCurr andRefPicSetLtFoll are marked as “used for long-term reference,” that is,“used as LTRPs.”

3. As illustrated below, an STRPS is derived, and an STRP is marked.

for( i = 0; i < NumPocStCurrBefore; i++ )↓ if( there is a short-termreference picture picX In the DPB↓ with PicOrderCntVal equal toPocStCurrBefore[ i ])↓ RefPicSetStCurrBefore[i ] = picX↓ else↓RefPicSetStCurrBefore[i ] = “no reference picture” 

for( i = 0; i < NumPocStCurrAfter; i++ )↓ if( there is a short-termreference picture picX in the DPB↓ with PicOrderCntVal equal toPocStCurrAfter[ i ])↓ RefPicSetStCurrAfter[ i ] = picX↓ else↓RefPicSetStCurrAfter[ i ] = “no reference picture” for( i = 0; i <NumPocStFoll; i++ )↓ if( there is a short-term reference picture picX inthe DPB↓ with PicOrderCntVal equal to PocStFoll[ i ])↓ RefPicSetStFoll[i ] = picX↓ else↓ RefPicSetStFoll[ i ] = “no reference picture” 

4. If TemporalId of the current picture is 0, all pictures included inthe DPB marked as “used for long-term reference,” not included inRefPicSetLtCurr and RefPicSetLtFoll, are marked as “unused forreference.”

5. Alternatively, if TemporalId of the current picture is not 0, allpictures included in the DPB marked as “used for long-term reference,”not included in RefPicSetLtCurr and RefPicSetLtFoll, are automaticallyincluded in RefPicSetLtFoll.

6. All reference pictures in the DPB marked as “used for short-termreference,” not included in RefPicSetLtCurr, RefPicSetLtFoll,RefPicSetStCurrBefore, RefPicSetStCurrAfter and RefPicSetStFoll, aremarked as “unused for reference.”

FIG. 9 illustrates a marking process according to the fourth embodiment.Each reference picture in the DBP may be marked according to the processof FIG. 9.

First, it is determined whether a first reference picture is marked asan STRP, that is, whether the first reference picture is an STRP (S910).

If the first reference picture is an STRP, operation S920 is carriedout. If the first reference picture is not an STRP, operation S970 iscarried out.

If the first reference picture is an STRP, it is determined whether thefirst reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll (S920).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, the first reference picture is marked as “used forlong-term reference” (S930), followed by operation S960.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, operation S940 is carried out.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, it is determined whether the first reference picture isincluded in RefPicSetStCurrBefore, RefPicSetStCurrAfter, orRefPicSetLtFoll (S940).

If the first reference picture is not included in RefPicSetStCurrBefore,RefPicSetStCurrAfter, and RefPicSetLtFoll, the first reference pictureis marked as “unused for reference” (S950), followed by operation S960.

If the first reference picture is included in RefPicSetStCurrBefore,RefPicSetStCurrAfter, or RefPicSetLtFoll, operation S960 is carried out.

If the first reference picture is not an STRP in operation S910, it isdetermined whether the first reference picture is included inRefPicSetLtCurr or RefPicSetLtFoll (S970).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, operation S960 is carried out. If the first referencepicture is not included in RefPicSetLtCurr and RefPicSetLtFoll,operation S980 is carried out.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, it is determined whether TemporalId of the currentpicture is 0 (S980).

If TemporalId of the current picture is 0, the first reference pictureis marked as “unused for reference” (S950), followed by operation S960.

If TemporalId of the current picture is not 0, operation S960 is carriedout.

It is determined whether more reference pictures are present in the DPBin operation S960. If an additional reference picture is present, a nextreference picture as a reference picture marking target is subjected tothe foregoing process to start operation S910.

If there is no reference picture as a reference picture marking target,the reference picture marking process terminates.

Derived RPSs and marked pictures for Pic 266 and Pic 268 of FIG. 6according to the present embodiment are illustrated in FIG. 7.

When processing Pic 267 having TemporalId which is not 0, an LTRP maynot be marked as “unused for reference,” and thus two reference pictureshaving the same POC LSB are present when processing Pic 268. Thus,according to FIG. 7, in a process of marking Pic 268, MSB informationfor an LTRP is signaled, and accordingly the foregoing problem does notoccur any more.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 252 is marked as “unused for            reference”

2. For Pic 268, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={256}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            long-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture POC 266 is marked as “used for short-term            reference”        -   Reference picture with POC 0 is marked as “unused for            reference”

Fifth Embodiment

According to a fifth embodiment of the present invention, if a currentpicture is not at a base temporal layer, that is, TemporalId is not 0,and is a picture that can be discarded without affecting decodability ofother pictures having the same TemporalId, an LTRP in the DPB is notallowed to be marked as “unused for reference.”

The fifth embodiment may be modified as follows.

When the current picture is at the base temporal layer, that is,TemporalId is 0, and it cannot be discarded without affectingdecodability of other pictures having the same TemporalId, an LTRP inthe DPB is marked as “unused for reference.”

According to the foregoing variation of the fifth embodiment, an RPS foran LTRP is derived as follows.

1. An LTRPS is derived first, and an LTRP is marked.

for( i = 0; i < NumPocLtCurr; i++ )↓ if( !CurrDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB ↓ withpic_order_cnt_lsb equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr [ i ] =picX↓ else if( there is a short-term reference picture picY in the DPBwith pic_order_cnt_lsb equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[i ] =picY↓ else ↓ RefPicSetLtCurr[i ] = “no reference picture”↓ else↓ if(there is a long-term reference picture picX in the DPB↓ withPicOrderCntVal equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[ i ] = picX↓else if( there is a short-term reference picture picY in the DPB withPicOrderCntVal equal to PocLtCurr[ i ] )↓ RefPicSetLtCurr[i ] = picY↓else ↓ RefPicSetLtCurr[ i ] = “no reference picture” 

for( i = 0; i < NumPocLtFoll; i++ )↓ if( !FollDeltaPocMsbPresentFlag[ i] )↓ if( there is a long-term reference picture picX in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓else if( there is a short-term reference picture picY in the DPB↓ withpic_order_cnt_lsb equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[ i ] =picY↓ else ↓ RefPicSetLtFoll[i ] = “no reference picture”↓ else↓ if(there is a long-term reference picture picX in the DPB↓ withPicOrderCntVal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picX↓ else if(there is a short-term reference picture picY in the DPB withPicOrderCntVal equal to PocLtFoll[ i ] )↓ RefPicSetLtFoll[i ] = picY↓else↓ RefPicSetLtFoll[i ] = “no reference picture” 

2. All reference pictures included in RefPicSetLtCurr andRefPicSetLtFoll are marked as “used for long-term reference,” that is,“used as LTRPs.”

3. As illustrated below, an STRPS is derived, and an STRP is marked.

for( i = 0; i < NumPocStCurrBefore; i++ )↓ if( there is a short-termreference picture picX In the DPB↓ with PicOrderCntVal equal toPocStCurrBefore[ i ])↓ RefPicSetStCurrBefore[i ] = picX↓ else↓RefPicSetStCurrBefore[i ] = “no reference picture” 

for( i = 0; i < NumPocStCurrAfter; i++ )↓ if( there is a short-termreference picture picX in the DPB↓ with PicOrderCntVal equal toPocStCurrAfter[ i ])↓ RefPicSetStCurrAfter[ i ] = picX↓ else↓RefPicSetStCurrAfter[ i ] = “no reference picture” for( i = 0; i <NumPocStFoll; i++ )↓ if( there is a short-term reference picture picX inthe DPB↓ with PicOrderCntVal equal to PocStFoll[ i ])↓ RefPicSetStFoll[i ] = picX ↓ else↓ RefPicSetStFoll[ i ] = “no reference picture” 

4. If TemporalId of the current picture is 0 and an NAL unit typethereof is none of TRAIL_N, TSA_N, STSA_N, RADL_N and RASL_N, allpictures included in the DPB marked as “used for long-term reference,”not included in RefPicSetLtCurr and RefPicSetLtFoll, are marked as“unused for reference.”

5. Alternatively, if TemporalId of the current picture is not 0 or theNAL unit type is one of TRAIL_N, TSA_N, STSA_N, RADL_N and RASL_N, allpictures included in the DPB marked as “used for long-term reference,”not included in RefPicSetLtCurr and RefPicSetLtFoll, are automaticallyincluded in RefPicSetLtFoll.

6. All reference pictures in the DPB marked as “used for short-termreference,” not included in RefPicSetLtCurr, RefPicSetLtFoll,RefPicSetStCurrBefore, RefPicSetStCurrAfter and RefPicSetStFoll, aremarked as “unused for reference.”

FIG. 10 illustrates a marking process according to the fifth embodiment.Each reference picture in the DBP may be marked according to the processof FIG. 10.

First, it is determined whether a first reference picture is marked asan STRP, that is, whether the first reference picture is an STRP(S1010).

If the first reference picture is an STRP, operation S1020 is carriedout. If the first reference picture is not an STRP, operation S1070 iscarried out.

If the first reference picture is an STRP, it is determined whether thefirst reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll (S1020).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, the first reference picture is marked as “used forlong-term reference” (S1030), followed by operation S1060.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, operation S1040 is carried out.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, it is determined whether the first reference picture isincluded in RefPicSetStCurrBefore, RefPicSetStCurrAfter, orRefPicSetLtFoll (S1040).

If the first reference picture is not included in RefPicSetStCurrBefore,RefPicSetStCurrAfter, and RefPicSetLtFoll, the first reference pictureis marked as “unused for reference” (S1050), followed by operationS1060.

If the first reference picture is included in RefPicSetStCurrBefore,RefPicSetStCurrAfter, or RefPicSetLtFoll, operation S1060 is carriedout.

If the first reference picture is not an STRP in operation S1010, it isdetermined whether the first reference picture is included inRefPicSetLtCurr or RefPicSetLtFoll (S1070).

If the first reference picture is included in RefPicSetLtCurr orRefPicSetLtFoll, operation S1060 is carried out. If the first referencepicture is not included in RefPicSetLtCurr and RefPicSetLtFoll,operation S1080 is carried out.

If the first reference picture is not included in RefPicSetLtCurr andRefPicSetLtFoll, it is determined whether TemporalId of the currentpicture is 0 and the NAL unit type is not a non-reference picture(S1080).

If TemporalId of the current picture is 0 and the NAL unit type is not anon-reference picture, the first reference picture is marked as “unusedfor reference” (S1050), followed by operation S1060.

If TemporalId of the current picture is not 0 or the NAL unit type is anon-reference picture, operation S1060 is carried out.

It is determined whether more reference pictures are present in the DPBin operation S1060. If an additional reference picture is present, anext reference picture as a reference picture marking target issubjected to the foregoing process to start operation S1010.

If there is no reference picture as a reference picture marking target,the reference picture marking process terminates.

Derived RPSs and marked pictures for Pic 266 and Pic 268 of FIG. 6according to the present embodiment are illustrated in FIG. 7.

When processing Pic 267 having TemporalId which is not 0 and an NAL unittype which is one of TRAIL_N, TSA_N, STSA_N, RADL_N and RASL_N, an LTRPmay not be marked as “unused for reference,” and thus two referencepictures having the same POC LSB are present when processing Pic 268.Thus, according to FIG. 7, in a process of marking Pic 268, MSBinformation for an LTRP is signaled, and accordingly the foregoingproblem does not occur any more.

1. For Pic 266, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={0}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={256, 264}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 0 is marked as “used for            long-term reference”        -   Reference picture with POC 256 is marked as “used for            short-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 252 is marked as “unused for            reference”

2. For Pic 268, an RPS is derived and a reference picture is marked asfollows.

-   -   RefPicSetLtCurr={256}    -   RefPicSetLtFoll={ }    -   RefPicSetStCurrBefore={264, 266}    -   RefPicSetStCurrAfter={ }    -   RefPicSetStFoll={ }    -   State of DPB:        -   Reference picture with POC 256 is marked as “used for            long-term reference”        -   Reference picture with POC 264 is marked as “used for            short-term reference”        -   Reference picture with POC 266 is marked as “used for            short-term reference”        -   Reference picture with POC 0 is marked as “unused for            reference”

Sixth Embodiment

According to a sixth embodiment of the present invention, when tworeference pictures having the same POC LSB are present, at least one ofwhich is an LTRP that may be or is marked as “unused for reference,” ifa new LTRP having the same POC LSB as that of the LTRP marked as “unusedfor reference” is present before a picture which has TemporalId not 0and is a picture that can be discarded without affecting decodability ofother pictures with the same TemporalId, information for calculating orinferring a POC MSB of the new LTRP may be necessarily signaled until apicture having TemporalId equal to 0 and discardable without affectingdecodability of other pictures having the same TemporalId is received orbeing included in the picture.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

The restriction may be applied to a semantic elementdelta_msb_poc_present_flag [i] controlling existence of the informationfor calculating or inferring the POC MSB of the LTRP.

Accordingly, the semantic element delta_poc_msb_present_flag[i] may beinterpreted as follows.

delta_poc_msb_present_flag[i] is 1 when at least one LTRP marked as“unused for reference” and having the same POC LSB as poc_lsb_lt[i] ispresent after a last picture in decoding order having TemporalId equalto 0 and an NAL unit type that is none of TRAIL_N, TSA_N, STSA_N, RADL_Nand RASL_N.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Seventh Embodiment

According to a seventh embodiment of the present invention, when tworeference pictures having the same POC LSB are present, at least one ofwhich is an LTRP that may be or is marked as “unused for reference,” ifa new LTRP having the same POC LSB as that of the LTRP marked as “unusedfor reference” is present before a picture as a random access point,such as clean random access (CRA), instantaneous decoding refresh (IDR)and broken link access (BLA) pictures, information for calculating orinferring a POC MSB of the new LTRP may be necessarily signaled untilthe picture as the random access point is received or being included inthe picture.

An IDR picture as an IDR NAL unit type is a randomly accessible picture,which may be a first picture or intermediate picture in a bit stream indecoding order. The NAL unit type of the IDR picture may be representedas IDR_W_RADL. If the IDR picture is not associated with a leadingpicture, the NAL unit type of the IDR picture may be represented asIDR_N_LP.

A CRA picture as a CRA NAL unit type is a randomly accessible picture,which may be a first picture or intermediate picture in a bit stream indecoding order. The CRA picture includes an I slice only. The CRApicture may be associated with a leading picture which may be decodableand a leading picture which may skip a decoding process. The leadingpicture which may skip the decoding process may use a reference picturenot present in the bit stream, and thus the leading picture may not beoutput by a decoder.

A BLA picture as a BLA NAL unit type is a randomly accessible picture,which may be a first picture or intermediate picture in a bit stream indecoding order. The BLA picture also includes an I slice only. Each BLApicture may start a new coded video sequence (CVS) and be subjected tothe same decoding process as the IDR picture.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Eighth Embodiment

According to an eighth embodiment of the present invention, when thefollowing conditions or situations are satisfied, all LTRPs need to besignaled or included in an RPS of a current slice or current picture.

The present embodiment is applied under the conditions that a temporallevel of the current picture is not 0, that is, TemporalId is not 0, orthe current picture can be discarded without affecting decodability ofother pictures having the same TemporalId and one or more pictureshaving the same POC LSB are present in the DPB, one of which is an LTRP.

Here, an LTRP unused by the current slice may be still be included inthe RPS, with a syntax element used_by_curr_pic_lt_flag[i] having avalue of 0.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

The following constrain may be applied to the RPS.

That is, the RPS is required to have bit stream conformance that ifTemporalId of the current picture/slice is not 0 or an NAL unit type ofthe current slice is one of TRAIL_N, TSA_N, STSA_N, RADL_N and RASL_N,and one or more reference pictures having the same POC LSB are presentin the DPB, at least one of which is an LTRP, all LTRPs are necessarilyincluded in the RPS of the current slice.

Alternatively, the RPS is required to have bit stream conformance thatif TemporalId of the current slice is not 0 or an NAL unit type of thecurrent slice is one of TRAIL_N, TSA_N, STSA_N, RADL_N and RASL_N, andone or more reference pictures having the same POC LSB are present inthe DPB, at least one of which is an LTRP, all LTRPs necessarily existor must be contained in the RPS of the current slice.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Ninth Embodiment

According to a ninth embodiment of the present invention, MSBinformation for calculating a POC of a specific LTRP needs to be presentwhen the following situations are satisfied.

The present embodiment is applied under the situations that referencepictures for a previous picture which has TemporalId equal to 0 and isnot discardable without affecting decodability of other pictures in thesame temporal layer, that is, has an NAL unit type being none ofTRAIL_N, TSA_N and STSA_N, include at least one reference picture havingthe same POC LSB as POC LSB of an LTRP for a current picture.

Here, the following mandate or restriction is applied.

A previous picture in decoding order which has an NAL unit type beingnone of TRAIL_N, TSA_N, STSA_N, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14and has TemporalId equal to 0 is set as “prevTid0Pic.”

A previous picture POC set (setOfPreviousPictures) including POCsrelating to prevTid0Pic are set to include all pictures present in anRPS of prevTid0Pic and all pictures following prevTid0Pic and precedingthe current picture in decoding order.

delta_poc_msb_present_flag[i] shall be equal to 1 when there is morethan one picture in setOfPreviousPictures with picture order countmodulo MaxPicOrderCntLsb equal to PocLsbLt[i].

That is, when prevTid0Pic and setOfPreviousPictures are set as above anda reference picture with a specific condition is present,delta_poc_msb_present_flag[i] is 1, delta_poc_msb_cycle_lt[i] fordetermining DeltaPocMsbCycleLt used for calculating a POC MSB of an LTRPof the current picture is signaled to the decoding apparatus.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Here, RSV_VCL_N10, RSV_VCL_N12 or RSV_VCL_N14 refers to a VCL NAL unittype reserved for a sub-layer non-reference picture, not an intra randomaccess point.

The sub-layer non-reference picture is a picture that cannot be used forinter prediction in decoding process a subsequent picture in decodingorder in the same sub-layer. A picture having an NAL unit type ofTRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N, RSV_VCL_N10, RSV_VCL_N12 orRSV_VCL_N14 is a sub-layer non-reference picture.

Tenth Embodiment

A tenth embodiment of the present invention is applied under thefollowing situations.

The present embodiment is applied under the situations that when one ormore reference pictures with the same POC LSB as a POC LSB (PocLsbLt[i])of an LTRP of the current picture are present in the DPB, starting froma next picture to a first picture following a current picture indecoding order and having TemporalId equal to 0 and an NAL unit typebeing none of TRAIL_N, TSA_N, STSA_N, RSV_VCL_N10, RSV_VCL_N12 andRSV_VCL_N14.

Here, the following mandate or restriction is applied.

When one or more reference pictures with POC modulo MaxPicOrderCntLsbequal to PocLsbLt[i] are present in the DPB,delta_poc_msb_present_flag[i] for an LTRP having a POC LSB the same asPocLsbLt [i] is 1 for all sequence pictures following the currentpicture in decoding order until and including a first picture followingthe current picture in decoding order and having TemporalId equal to 0and an NAL unit type being none of TRAIL_N, TSA_N, STSA_N, RADL_N,RASL_N, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Eleventh Embodiment

An eleventh embodiment of the present invention is a variation of theninth embodiment.

According to the present embodiment, MSB information for calculating aPOC of a specific LTRP needs to be present when the following situationsare satisfied.

The present embodiment is applied under the conditions that referencepictures of a previous picture which has TemporalId equal to 0 and isnot discardable without affecting decodability of other pictures in thesame temporal layer, that is, has an NAL unit type not being any one ofTRAIL_N, TSA_N and STSA_N, include at least one reference picture havingthe same POC LSB as POC LSB of an LTRP for a current picture.

Here, the following mandate or restriction is applied.

Let “prevTid0Pic” be a previous picture in decoding order which has anNAL unit type being none of TRAIL_N, TSA_N, STSA_N, RSV_VCL_N10,RSV_VCL_N12 and RSV_VCL_N14 and has TemporalId equal to 0 and letsetOfPreviousPictures be the set consisting of all pictures present inan RPS of prevTid0Pic and all pictures following prevTid0Pic andpreceding the current picture in decoding order.

delta_poc_msb_present_flag[i] is 1 when one or more pictures with POCmodulo MaxPicOrderCntLsb equal to PocLsbLt[i] are present insetOfPreviousPictures.

Alternatively, prevTid0Pic is a previous picture in decoding order whichhas an NAL unit type being none of TRAIL_N, TSA_N, STSA_N, RADL_N,RASL_N, RASL_R, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14 and TemporalIdequal to 0.

SetOfPreviousPictures is set to include prevTid0Pic, all picturespresent in the RPS of prevTid0Pic, and all pictures present in an RPS ofall pictures following prevTid0Pic and preceding the current picture indecoding order.

delta_poc_msb_present_flag[i] is 1 when one or more pictures with POCmodulo MaxPicOrderCntLsbg equal to PocLsbLt[i] are present insetOfPreviousPictures.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Twelfth Embodiment

A twelfth embodiment of the present invention is a variation of theninth embodiment.

According to the present embodiment, MSB information for calculating aPOC of a specific LTRP needs to be present when the following situationsare satisfied.

The present embodiment is applied under the situations that referencepictures of a previous picture that has TemporalId equal to 0 and is notdiscardable without affecting decodability of other pictures in the sametemporal layer include at least one reference picture having the samePOC LSB as that of an LTRP for a current picture.

That is, the present embodiment may be applied when one or morereference pictures having the same POC LSB as POC LSB of the LTRP forthe current picture are present in a case where the previous picture hasan NAL unit type being none of TRAIL_N, TSA_N and STSA_N and hasTemporalId equal to 0. Here, the following mandate or restriction isapplied.

A previous picture in decoding order which has TemporalId equal to 0 andcannot be discarded without affecting decodability of other pictures isset as “prevTid0Pic.”

For example, prevTid0Pic refers to a picture in decoding order which hasan NAL unit type being none of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N,RASL_R, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14 and TemporalId equal to0.

prevTid0Pic is not an RASL, RADR or sub-layer non-reference picturewhich can be discarded without affecting decodability of other pictures.

setOfPreviousPictures is set to include a POC of prevTid0Pic, a POC ofeach picture present in an RPS of prevTid0Pic, and a POC of each picturefollowing prevTid0Pic and preceding the current picture in decodingorder.

Alternatively, setOfPreviousPictures may be set to include a POC ofprevTid0Pic, POCs of all pictures present in the RPS of prevTid0Pic andPOCs of all pictures present in an RPS for all pictures followingprevTid0Pic and preceding the current picture in decoding order.

Alternatively, setOfPreviousPictures may be set to include POCs of allpictures present in the RPS of prevTid0Pic and POCs of all picturesfollowing prevTid0Pic and preceding the current picture in decodingorder.

delta_poc_msb_present_flag[i] is 1 when one or more pictures with POCmodulo MaxPicOrderCntLsb equal to PocLsbLt[i] are present insetOfPreviousPictures.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Thirteenth Embodiment

A thirteenth embodiment of the present invention is a variation of thetenth embodiment.

When one or more pictures with POC modulo MaxPicOrderCntLsb equal toPocLsbLt[i] are present in the DPB, delta_poc_msb_present_flag[i] is 1for a current picture and for all sequence pictures following thecurrent picture until and including a first picture following thecurrent picture in decoding order and having TemporalId equal to 0 andan NAL unit type being none of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N,RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14.

If PicOrderCnt(currPic) & (MaxPicOrderCntLsb−1) is equal toPicOrderCnt(picX) & (MaxPicOrderCntLsb−1) for any reference picture picXin the DPB, delta_poc_msb_present_flag[i] is 1 for all sequence picturesfollowing the current picture in decoding order until and including thefirst picture following the current picture in decoding order and havingTemporalId equal to 0 and an NAL unit type being none of TRAIL_N, TSA_N,STSA_N, RADL_N, RASL_N, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Fourteenth Embodiment

A fourteenth embodiment of the present invention is a variation of thethirteenth embodiment.

When one or more picture with POC modulo MaxPicOrderCntLsb equal toPocLsbLt[i] are present in the DPB, delta_poc_msb_present_flag[i] is 1for a current picture and for all sequence pictures following thecurrent picture until and including a first picture following thecurrent picture in decoding order and having TemporalId of 0 and an NALunit type being none of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N,RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14.

If PicOrderCnt(currPic) & (MaxPicOrderCntLsb−1) is equal to PocLsbLt[i],delta_poc_msb_present_flag[i] is 1 for all sequence pictures followingthe current picture in decoding order until and including the firstpicture following the current picture in decoding order and havingTemporalId equal to 0 and an NAL unit type being none of TRAIL_N, TSA_N,STSA_N, RADL_N, RASL_N, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Fifteenth Embodiment

A fifteenth embodiment of the present invention is a variation of thetwelfth embodiment.

According to the present embodiment, MSB information for calculating aPOC of a specific LTRP needs to be present when the following situationsare satisfied.

The present embodiment is applied under the situations that at least onereference picture having the same POC LSB as that of an LTRP for acurrent picture is present with respect to a reference picture of aprevious picture named prevKeyPicture, which has TemporalId equal to 0and is not discardable without affecting other pictures in the sametemporal layer ID, prevKeyPicture itself, and all pictures followingprevKeyPicture and preceding the current picture in decoding order.

That is, the present embodiment may be applied when at least onereference picture having the same POC LSB as that of the LTRP for thecurrent picture is present in a case where the previous picture has anNAL unit type being none of TRAIL_N, TSA_N and STSA_N and TemporalIdequal to 0.

Here, the following mandate or restriction is applied.

A previous picture in decoding order which has an NAL unit type beingnone of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N, RASL_R, RSV_VCL_N10,RSV_VCL_N12 and RSV_VCL_N14 and TemporalId equal to 0 is set as“prevTid0Pic.”

setOfPreviousPictures is set to include PicOrderCntVal of prevTid0Pic,PicOrderCntVal of each picture present in an RPS of prevTid0Pic, andPicOrderCntVa of each picture following prevTid0Pic and preceding thecurrent picture in decoding order.

Here, a picture having an NAL unit type which is none of TRAIL_N, TSA_N,STSA_N, RADL_N, RASL_N, RASL_R, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14may also be expressed as picture that is neither a sub-layernon-reference picture nor a RASL picture.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

Sixteenth Embodiment

A sixteenth embodiment of the present invention is a variation of thefifteenth embodiment.

According to the present embodiment, MSB information for calculating aPOC of a specific LTRP needs to be present when the following situationsare satisfied.

The present embodiment is applied under the situations that referencepictures of a previous picture having TemporalId equal to 0 and an NALunit type not discardable without affecting decodability of otherpictures in the same temporal layer, that is, the NAL unit type beingnone of TRAIL_N, TSA_N and STSA_N, include at least one referencepicture having the same POC LSB as that of an LTRP of a current picture.

Here, the following mandate or restriction is applied.

prevTid0Pic, which represents a previous picture at a front position indecoding order having TemporalId equal to 0, has an NAL unit type beingnone of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N, RASL_R, RSV_VCL_N10,RSV_VCL_N12 and RSV_VCL_N14 and has TemporalId equal to 0.

setOfPreviousPictures is set to include PicOrderCntVal of prevTid0Pic,PicOrderCntVal of each picture present in an RPS of prevTid0Pic, andPicOrderCntVal of each picture following prevTid0Pic and preceding thecurrent picture in decoding order as a non-reference picture havingTemporalId being not the same as HighestTid.

delta_poc_msb_present_flag[i] is 1 when one or more pictures with POCmodulo MaxPicOrderCntLsb equal to PocLsbLt[i] are present insetOfPreviousPictures.

Here, a picture having an NAL unit type being none of TRAIL_N, TSA_N,STSA_N, RADL_N, RASL_N, RASL_R, RSV_VCL_N10, RSV_VCL_N12 and RSV_VCL_N14may also be expressed as picture that is neither a sub-layernon-reference picture nor RASL picture.

According to the present embodiment, the marking process is not changedexcept that the foregoing mandate or restriction is added, and theencoding apparatus may need to comply with the mandate or restriction.

According to the present embodiment, information for calculating orinferring an MSB of a POC of 256 is signaled when marking a picture witha POC of 268.

FIG. 11 is a flowchart illustrating a process of deriving an RPS andmarking a picture according to the present invention.

The decoding apparatus receives RPS information for constructing an RPSof a current picture (S1110).

The RPS information may include information on a number of STRPSs, indexinformation on an STRPS of the current picture, and POC differenceinformation for calculating POCs of STRPs forming the STRPS.

Further, the RPS information may include flag information indicatingexistence of an LTRP, information on a number of LTRPs, indexinformation on an STRP of the current picture, information on an LSB ofan LTRP, MSB information for calculating an MSB of a POC of an LTRP, andflag information indicating whether MSB information is present, that is,MSB information is received. The RPS information may be received in abit stream relating to an SPS, PPS or slice header.

The MSB information on the LTRP is received only when the flaginformation indicating existence of MSB information, for example,delta_poc_msb_present_flag, is 1.

The MSB information may be MSB cycle information for determining a valuecorresponding to a difference in POC MSB between the current picture andthe LTRP, for example, DeltaPocMsbCycleLt, and be signaled with a syntaxelement, for example, delta_poc_msb_cycle_lt.

The difference in POC MSB between the current picture and the LTRPderived corresponding to a signaled value of delta_poc_msb_cycle_lt maybe derived as DeltaPocMsbCycleLt for first and last LTRPs that is thesignaled value of delta_poc_msb_cycle_lt.

DeltaPocMsbCycleLt[i] for an i-th LTRP, other than the first and lastLTRPs, may be derived as a sum of signaled delta_poc_msb_cycle_lt[i] andDeltaPocMsbCycleLt[i−1] for a previous LTRP.

Here, the MSB information on the LTRP may be present when a specificcondition is satisfied. That is, delta_poc_msb_present_flag of 1 issignaled under the specific condition.

When a previous picture that has a temporal sub-layer ID (TemporalId)equal to 0 and cannot be discarded without affecting decodability ofother pictures in the same temporal sub-layer is set as prevTid0Pic, aprevious picture POC set, setOfPrevPocVals, including POCs relating toprevTid0Pic may be constructed.

the previous picture prevTid0Pic has an NAL unit type of not equal toone of TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N, RSV_VCL_N10, RSV_VCL_N12and RSV_VCL_N14.

Alternatively, the previous picture prevTid0Pic may not be a sub-layernon-reference picture not used for inter prediction inter prediction inthe decoding process of subsequent pictures of the same sub-layer indecoding order.

The previous picture POC set setOfPrevPocVals may include a POC of theprevious picture, a POC of each reference picture in an RPS of theprevious picture, and a POC of each picture following the previouspicture in decoding order and preceding the current picture in decodingorder.

To sum up, according to the present invention, the MSB information forcalculating the MSB of the POC of the LTRP for the current picture issignaled when the previous picture POC set includes at least one POChaving the same LSB as that of the LTRP for the current picture, thatis, reference pictures for the previous picture include a referencepicture having the same LSB as that of the LTRP for the current picture.

Here, the previous picture is a picture which has TemporalId equal to 0and cannot be discarded without affecting decodability of other picturesin the same temporal sub-layer.

Referring to FIG. 6, even though Pic 267 is removed, if a previouspicture POC set constructed for a previous picture Pic 266 includes areference picture having the same POC LSB as that of the LTRP for thecurrent picture, delta_poc_msb_present_flag of 1 is signaled for Pic268. Thus, an LTRPS for Pic 268 may be normally derived.

After receiving and parsing the RPS information, the decoding apparatusconstructs a POC list for deriving the RPS (S1120).

The POC list may include an STRP POC list including POCs of STRPs of thecurrent picture and an LTRP POC list including POCs of LTRPs of thecurrent picture.

The STRP POC list may include a forward STRP POC list PocStCurrBefore, abackward STRP POC list PocStCurrAfter, and a non-reference STRP POC listPocStFoll.

The LTRP POC list may include an LTRP POC list PocLtCurr used by thecurrent picture and an LTRP POC list PocLtFoll not used by the currentpicture.

An STRP POC forming the POC list is generated using a difference in POCbetween the current picture and an STRP.

That is, POCs of reference pictures included in the STRP POC list may bedetermined based on relative POCs. Here, information on the relativePOCs may be transmitted from the encoding apparatus to the decodingapparatus.

A relative POC may represent a POC difference between two pictures in anRPS. Relative POCs of reference pictures before the current picture inPOC order, that is, reference pictures having a smaller POC than that ofthe current picture, may correspond to POC differences from referencepictures right before the reference pictures in the RPS. Relative POCsof reference pictures after the current picture in POC order, that is,reference pictures having a greater POC than that of the currentpicture, may correspond to POC differences from reference pictures rightbefore the reference pictures in the RPS.

In the forward STRP POC list, forward STRPs having a smaller POC thatthat of the current picture may be disposed in descending order of POCs.That is, pictures having a smaller POC than that of the current pictureamong pictures in the DPB may be disposed in descending order of POCsfrom a start of the forward STRP POC list.

In the backward STRP POC list, backward STRPs having a greater POC thatthat of the current picture may be disposed in ascending order of POCs.That is, pictures having a greater POC than that of the current pictureamong the pictures in the DPB may be disposed in ascending order of POCsfrom a start of the backward STRP POC list.

An STRP may be included in PocStFoll or PocStCurrBefore/PocStCurrAftercorresponding to information indicating whether the STRP is used by thecurrent picture, such as used_by_curr_pic_s0_flag orused_by_curr_pic_s1_flag.

Meanwhile, a POC of an LTRP is generated by an operation that is POC ofcurrent picture−DeltaPocMsbCycleLt*MaxPicOrderCntLsb−POC LSB of currentpicture+POC LSB of LTRP. Here, DeltaPocMsbCycleLt has a valuecorresponding to a difference in POC MSB between the current picture andthe LTRP.

FIG. 12 illustrates a method of deriving a POC of an LTRP.

As shown in FIG. 12, an LTRP has a great POC difference from the currentpicture and thus may be expressed using a POC LSB and a POC MSB.

MaxPocLsb refers to a maximum LSB value. For example, if MaxPocLsb is32, an LTRP (LTRF) with a POC of 84 may be expressed as 32*2+20, inwhich LSB is 20 and MSB is 2.

The current picture with a POC of 338 may be expressed as 32*10+11 whena maximum value expressed in LSB is 32, in which MSB is 10 and LSB is11.

delta_poc_msb_cycle_lt is a value for determining a POC MSB, that is,DeltaPocMsbCycleLt, in an LTRPS of the current picture.DeltaPocMsbCycleLt may correspond to a difference in POC MSB between thecurrent picture and a reference picture.

A POC of an LTRP may be derived using a POC LSB of the reference pictureand a difference in POC MSB between the current picture and thereference picture.

For example, when the current picture has a POC of 331, an LTRP with aPOC of 308 indexed to 0, LTRP[0], may be expressed as 331−1*32−11+20using an LSB of the LTRP of 20 and a difference in POC MSB between thecurrent picture and the LTRP equal to 1.

Similarly, an LTRP with a POC of 170 indexed to 1, LTRP [1], may beexpressed as 331−5*32−11+10 using an LSB of the LTRP of 10 and adifference in POC MSB between the current picture and the LTRP equal to5.

An LTRP may be included in PocLtFoll or PocLtCurr corresponding toinformation indicating whether the reference picture is used by thecurrent picture, such as used_by_curr_pic_lt_flag.

When the POC list is generated, the RPS is derived using the POC list,and a reference picture is marked (S1130).

If a picture with POC LSB or POC equal to a POC in the LTRP POC list ispresent in the DPB storing a reconstructed picture, the decodingapparatus includes the picture in an LTRPS RefPicSetLtCurr orRefPicSetLtFoll.

If a picture with POC LSB or POC equal to the POC in the LTRP POC listis not present in the DPB, the RPS includes “no reference picture,” thatis, no reference picture is present in an entry.

All pictures in the LTRPS are marked as “used for long-term reference.”

Subsequently, if a picture with POC equal to a POC in the STRP POC list,the decoding apparatus includes the picture in an STRPS.

If a picture with the same POC as the POC in the STRP POC list is notpresent in the DPB, the RPS includes “no reference picture,” that is, noreference picture is present in an entry.

All pictures in the DBP not included in the LTRPS and STRPS are markedas “unused for reference.” The picture marked as “unused for reference”may be removed from the DPB.

According to the present invention, in marking a reference picture,reference pictures in the POC list are marked as “unused for reference”or “used for long-term reference.”

FIG. 13 is a flowchart illustrating a video decoding method according tothe present invention.

First, the decoding apparatus receives information on a video, such asslice type information on a current picture, RPS information forconstructing an RPS of the current picture and prediction modeinformation on the current picture (S1310).

The decoding apparatus determines based on the prediction modeinformation whether a prediction target block is subjected to intraprediction or inter prediction (S1320).

If intra prediction is applied to the prediction target block, thedecoding apparatus generates a reconstructed picture via a process ofderiving an intra prediction mode and a process of generating aprediction block (S1330).

If inter prediction is applied to the prediction target block, thedecoding apparatus derives an RPS and derives a reference picture listbased on the RPS (S1340). The RPS is derived when a first slice of thecurrent picture is not an IDR picture. A process of deriving the RPS andmarking a picture is substantially the same as that illustrated in FIG.11, and thus description thereof is omitted herein.

A process of constructing the reference picture list to be illustratedbelow may be considered as a process of initializing the referencepicture list.

When reference picture list L0 is constructed, the decoding apparatusmay sequentially allocate reference picture indices to forward STRPsforming a forward STRPS, backward STRPs forming a backward STRPS andLTRPs forming an LTRPS, thereby constructing the reference picture list.That is, in L0, the forward STRPs may be allocated, the backward STRPsmay be added, and then the LTRPs may be finally added.

The forward STRPs forming the forward STRPS may be added to L0 in thesame order as included in the forward STRPS. That is, the forward STRPsmay be disposed in descending order of POCs in L0, and a greaterreference index value may be allocated to a picture with a smaller POC.

The backward STRPs forming the backward STRPS may be added to L0 in thesame order as included in the backward STRPS. That is, the backwardSTRPs may be disposed in ascending order of POCs in L0, and a greaterreference index value may be allocated to a picture with a greater POC.

In addition, the LTRPs forming the LTRPS may be added to L0 in the sameorder as included in the LTRPS.

For a B slice, reference picture list L1 may be also generated inaddition to L0. When L1 is constructed, the decoding apparatus maysequentially allocate reference picture indices to the backward STRPsforming the backward STRPS, the forward STRPs forming the forward STRPSand the LTRPs forming the LTRPS, thereby constructing the referencepicture list. That is, in L1, the backward STRPs may be allocated, theforward STRPs may be added, and then the LTRPs may be finally added.

The backward STRPs forming the backward STRPS may be added to L1 in thesame order as included in the backward STRPS. That is, the backwardSTRPs may be disposed in ascending order of POCs in L1, and a greaterreference index value may be allocated to a picture with a greater POC.

The forward STRPs forming the forward STRPS may be added to L1 in thesame order as included in the forward STRPS. That is, the forward STRPsmay be disposed in descending order of POCs in L1, and a greaterreference index value may be allocated to a picture with a smaller POC.In addition, the LTRPs forming the LTRPS may be added to L1 in the sameorder as included in the LTRPS.

The reference pictures added to L0 and L1 may be sequentially allocatedreference picture indices.

A number of reference pictures to be included in the RPS may bedetermined based on information transmitted from the encoding apparatus.For instance, the encoding apparatus may construct a reference picturelist, determine a number of reference pictures to use, and transmitinformation on the number of reference pictures to use, for example,num_ref_idx_1X_default_active_minus1, X=0 or 1, as a syntax element of asequence parameter set (SPS) to the decoding apparatus. The decodingapparatus may use a number of reference pictures specified by a value ofnum_ref_idx_1X_default_active_minus1 plus 1 as a default in a currentsequence.

Further, to specify a number of reference pictures by each picture orslice, the encoding apparatus may transmit extra information indicatinga number of reference pictures, for example,num_ref_idx_l1_active_minus1, X=0 or 1, through a picture parameter set(PPS) or slice header. The decoding apparatus may adopt a value ofnum_ref_idx_l1_active_minus1 plus 1 as a number of reference picturesfor a current picture or current slice.

In the foregoing process, the reference picture lists may be consideredto be implicitly derived. When the reference picture lists areimplicitly derived, the encoding apparatus and the decoding apparatusmay derive the reference picture lists available for inter prediction ofthe current picture based on the POCs of the pictures as describedabove.

Meanwhile, the decoding apparatus may modify the implicitly derivedreference picture lists based on information explicitly transmitted fromthe encoding apparatus. Here, the encoding apparatus may transmit bothreference picture list modification information indicating that theimplicitly derived reference picture lists are modified and entryinformation indicating a specific entry forming the reference picturelists. When the reference picture lists are modified based on theinformation explicitly transmitted from the encoding apparatus to befinally specified, the reference picture lists may be considered to beexplicitly specified.

When L0 is explicitly specified, the encoding apparatus may transmitentry information on L0. The entry information on L0 may indicate areference picture corresponding to an index on L0. When L1 is explicitlyspecified, the encoding apparatus may transmit entry information on L1.The entry information on L1 may indicate a reference picturecorresponding to an index on L1.

For example, when the reference picture lists are explicitly specifiedby the entry information, order and/or reference picture indices of theforward STRPs, the backward STRPs and the LTRPs in the reference picturelists may be different from those in the implicitly derived referencepicture lists. Furthermore, when the reference picture lists arespecified by the entry information, available reference pictures to beutilized may be different from those in the implicitly derived referencepictures lists.

When the reference picture lists are explicitly specified, the decodingapparatus may construct the same reference picture lists as thoseconstructed by the encoding apparatus based on the reference picturelist modification information and the entry information.

In the foregoing method of implicitly deriving the reference picturelists, the RPS and the reference picture list are illustrated onlyconsidering available pictures for convenience of description, but anRPS and/or a reference picture list may be also constructed by theencoding apparatus and the decoding apparatus in view of whetherreference pictures are available or used.

When the reference picture list is derived, the decoding apparatusderives a predictive sample value of the prediction target block usingthe reference picture list (S1350) and generates a reconstructed picturebased on the predictive sample value (S1360).

Subsequently, the decoding apparatus marks the reconstructed picture as“used for short-term reference” (S1370).

According to the present invention, the reconstructed picture is markedas “used for short-term reference” after the reconstructing process, andis newly marked with respect to whether the picture is used as an LTRPand used as a reference picture by the process of deriving the RPS andmarking the picture when decoding a next picture.

Although methods of illustrative systems have been described with aseries of stages or blocks based on the flowcharts, the presentinvention is not limited to the foregoing sequence of the stages. Somestages may be carried out in different order from described above or atthe same time. Further, it should be noted that as the aforementionedembodiments may include various aspects of examples, combinations of theembodiments may be also understood as exemplary embodiments of thepresent invention. Thus, it will be appreciated by those skilled in theart that changes, modifications and alternatives may be made in theseexemplary embodiments without departing from the principles and spiritof be the invention, the scope of which is defined in the appendedclaims and their equivalents.

What is claimed is:
 1. A video decoding method, by a decoding apparatus,the method comprising: receiving reference picture set (RPS) informationfor a current picture, wherein the RPS information comprises flaginformation indicating whether most significant bit (MSB) informationfor calculating an MSB of a picture order count (POC) of a long-termreference picture (LTRP) for the current picture is present, and whereinthe RPS information further comprises the MSB information when a valueof the flag information is equal to 1; deriving the POC of the LTRPbased on the MSB information when the value of the flag information isequal to 1, wherein the value of the flag information is equal to 1 whenthere is more than one POC value, in a previous picture POC set, forwhich the POC value modulo MaxPicOrderCntLsb is equal to a leastsignificant bit (LSB) of the POC of the LTRP, MaxPicOrderCntLsb is apredetermined max least significant bit value, wherein the previouspicture POC set is derived based on a previous picture of the currentpicture in decoding order, and the previous picture has a temporalsub-layer identifier equal to 0, and wherein the previous picture POCset derived based on the previous picture of the current pictureincludes 1) a POC of each reference picture in an RPS for the previouspicture in decoding order, and 2) a POC of each picture following theprevious picture in decoding order and preceding the current picture indecoding order; configuring an RPS for the current picture based on thePOC of the LTRP; performing inter prediction for a block in the currentpicture based on the RPS for the current picture; and reconstructing thecurrent picture based on the result of the inter prediction.
 2. Themethod of claim 1, wherein the previous picture cannot be discardedwithout affecting decodability of another picture in the same temporallayer, and wherein the previous picture has a network abstraction layer(NAL) unit type not equal to TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N,RASL_R, RSV_VCL_N10, RSV_VCL_N12, or RSV_VCL_N14.
 3. The method of claim1, wherein the previous picture is not a sub-layer non-reference picturewhich is not used for inter prediction in the decoding process ofsubsequent pictures of the same sub-layer in decoding order.
 4. Themethod of claim 1, wherein the previous picture POC set further includea POC of the previous picture.
 5. The method of claim 1, wherein the MSBinformation is MSB cycle information determining a value correspondingto a difference in POC MSB between the current picture and the LTRP. 6.The method of claim 1, further comprising constructing a POC list forderiving the RPS, wherein the POC list comprises a short-term referencepicture (STRP) POC list comprising a POC of an STRP for the currentpicture and an LTRP POC list comprising the POC of the LTRP for thecurrent picture, the STRP POC list is generated using a difference inPOC between the current picture and the STRP, and the LTRP POC list isgenerated by an operation that is POC of currentpicture−DeltaPocMsbCycleLt*MaxPicOrderCntLsb−POC LSB of currentpicture+POC LSB of LTRP, DeltaPocMsbCycleLt having a value correspondingto a difference in POC MSB between the current picture and the LTRP. 7.The method of claim 6, wherein the deriving the RPS comprises:constructing an LTRP set using a picture with a POC LSB or POC equal toa POC in the LTRP POC list when there is the picture in a decodedpicture buffer (DPB) storing a reconstructed picture; marking allpictures in the LTRP set as “used for long-term reference;” constructingan STRP set using a picture with a POC equal to a POC in the STRP POClist when there is the picture in the DBP; and marking all referencepictures in the DPB which are not included on the LTRP set and the STRPset as “unused for reference”.
 8. The method of claim 1, furthercomprising: receiving slice type information on the current picture andprediction mode information on the current picture, wherein thereconstructing the current picture comprises: determining based on theprediction mode information whether a prediction target block issubjected to intra prediction or inter prediction; deriving a referencepicture list based on the RPS when inter prediction is applied to theprediction target block; deriving a predictive sample value of theprediction target block using the reference picture list; and generatinga reconstructed picture based on the predictive sample value of theprediction target block.
 9. The method of claim 8, wherein thereconstructed picture is marked as “used for short-term reference”. 10.A decoding apparatus for video decoding, the apparatus comprising: adecoder configured to receive reference picture set (RPS) information ffor a current picture, wherein the RPS information comprises flaginformation indicating whether most significant bit (MSB) informationfor calculating an MSB of a picture order count (POC) of a long-termreference picture (LTRP) for the current picture is present and whereinthe RPS information further comprises the MSB information when a valueof the flag information is equal to 1, to derive the POC of the LTRPbased on the MSB information when the value of the flag information isequal to 1, to configure an RPS for the current picture based on the POCof the LTRP, to perform inter prediction for a block in the currentpicture based on the RPS for the current picture, and to reconstruct thecurrent picture based on the result of the inter prediction; and amemory configured to store the reconstructed current picture, whereinthe value of the flag information is equal to 1 when there is more thanone POC value, in a previous picture POC set, for which the POC valuemodulo MaxPicOrderCntLsb is equal to a least significant bit (LSB) ofthe POC of the LTRP, MaxPicOrderCntLsb is a predetermined max leastsignificant bit value, wherein the previous picture POC set is derivedbased on a previous picture of the current picture in decoding order,and the previous picture has a temporal sub-layer identifier equal to 0,and wherein the previous picture POC set derived based on the previouspicture of the current picture includes 1) a POC of each referencepicture in an RPS for the previous picture in decoding order, and 2) aPOC of each picture following the previous picture in decoding order andpreceding the current picture in decoding order.
 11. The apparatus ofclaim 10, wherein the previous picture cannot be discarded withoutaffecting decodability of another picture in the same temporal layer,and wherein the previous picture has a network abstraction layer (NAL)unit type not equal to TRAIL_N, TSA_N, STSA_N, RADL_N, RASL_N, RASL_R,RSV_VCL_N10, RSV_VCL_N12, or RSV_VCL_N14.
 12. The apparatus of claim 10,wherein the previous picture is not a sub-layer non-reference picturewhich is not used for inter prediction in the decoding process ofsubsequent pictures of the same sub-layer in decoding order.
 13. Theapparatus of claim 10, wherein the previous picture POC set furtherinclude a POC of the previous picture.
 14. The apparatus of claim 10,wherein the MSB information is MSB cycle information determining a valuecorresponding to a difference in POC MSB between the current picture andthe LTRP.
 15. The apparatus of claim 10, wherein the decoder constructsa POC list for deriving the RPS, wherein the POC list comprises ashort-term reference picture (STRP) POC list comprising a POC of an STRPfor the current picture and an LTRP POC list comprising the POC of theLTRP for the current picture, the STRP POC list is generated using adifference in POC between the current picture and the STRP, and the LTRPPOC list is generated by an operation that is POC of currentpicture−DeltaPocMsbCycleLt*MaxPicOrderCntLsb−POC LSB of currentpicture+POC LSB of LTRP, DeltaPocMsbCycleLt having a value correspondingto a difference in POC MSB between the current picture and the LTRP. 16.The apparatus of claim 15, wherein the decoder constructs an LTRP setusing a picture with a POC LSB or POC equal to a POC in the LTRP POClist when there is the picture in a decoded picture buffer (DPB) storinga reconstructed picture, marks all pictures in the LTRP set as “used forlong-term reference”, constructs an STRP set using a picture with a POCequal to a POC in the STRP POC list when there is the picture in theDBP, and marks all reference pictures in the DPB which are not includedon the LTRP set and the STRP set as “unused for reference”.
 17. Theapparatus of claim 10, wherein the decoder receives slice typeinformation on the current picture and prediction mode information onthe current picture, wherein the decoder reconstructs the currentpicture by determining based on the prediction mode information whethera prediction target block is subjected to intra prediction or interprediction, by deriving a reference picture list based on the RPS wheninter prediction is applied to the prediction target block, by derivinga predictive sample value of the prediction target block using thereference picture list, and by generating a reconstructed picture basedon the predictive sample value of the prediction target block.
 18. Avideo encoding method, by an encoding apparatus, the method comprising:deriving a picture order count (POC) of a long-term reference picture(LTRP) for a current picture; setting reference picture set (RPS)information for the current picture, wherein the RPS informationcomprises flag information indicating whether most significant bit (MSB)information for calculating an MSB of the POC of the LTRP for thecurrent picture is present, wherein the RPS information further comprisethe MSB information when a value of the flag information is equal to 1;encoding video information including the RPS information for the currentpicture; transmitting the encoded video information; configuring an RPSfor the current picture based on the POC of the LTRP; performing interprediction for a block in the current picture based on the RPS for thecurrent picture; and reconstructing the current picture based on theresult of the inter prediction; storing the reconstructed currentpicture in a memory, wherein the value of the flag information is setequal to 1 when there is more than one POC value, in a previous picturePOC set, for which the POC value modulo MaxPicOrderCntLsb is equal to aleast significant bit (LSB) of the POC of the LTRP, MaxPicOrderCntLsb isa predetermined max least significant bit value, wherein the previouspicture POC set is derived based on a previous picture of the currentpicture in decoding order, and the previous picture has a temporalsub-layer identifier equal to 0, and wherein the previous picture POCset derived based on the previous picture of the current pictureincludes 1) a POC of each reference picture in an RPS for the previouspicture in decoding order, and 2) a POC of each picture following theprevious picture in decoding order and preceding the current picture indecoding order.
 19. The method of claim 18, wherein the previous pictureis not a sub-layer non-reference picture which is not used for interprediction in the decoding process of subsequent pictures of the samesub-layer in decoding order.
 20. The method of claim 18, wherein theprevious picture POC set further include a POC of the previous picture.